Reading additional input from stdin... OpenAI Codex v0.124.0 (research preview) -------- workdir: /opt/MM_D-ND model: gpt-5.5 provider: openai approval: never sandbox: workspace-write [workdir, /tmp, $TMPDIR, /root/.codex-lab/memories] reasoning effort: none reasoning summaries: none session id: 019dfbf6-86a0-7340-9506-050e866c5b89 -------- user Sei l'AI-Lab D-ND. Leggi tools/data/agent_field_live.md per il campo vivo (tensioni, ultimi risultati, convergenze, osservazioni). Leggi tools/LAB_AGENT_CONTEXT.md per modello, regole, errori da evitare. Esegui un esperimento e scrivi il report in tools/data/reports/agent_20260506_0625.md warning: Codex could not find bubblewrap on PATH. Install bubblewrap with your OS package manager. See the sandbox prerequisites: https://developers.openai.com/codex/concepts/sandboxing#prerequisites. Codex will use the vendored bubblewrap in the meantime. codex Leggo prima il contesto operativo e il “campo vivo”, poi scelgo un esperimento coerente con le ultime convergenze e lo documento nel percorso richiesto. exec /bin/bash -lc "sed -n '1,240p' tools/data/agent_field_live.md" in /opt/MM_D-ND succeeded in 0ms: Leggi tools/LAB_AGENT_CONTEXT.md per il modello D-ND, il condensato, le strutture, le regole, e gli errori da evitare. ## Run precedente: completato (522s) — parti dalla consecutio. _Affinatore del run precedente: pending (affinatore empty output (exit 0))_ ## Piano 65 — Esplorare il confine: 8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo ## Tensioni attive - [TRASCENDENZA_LIMITE] (0.9) La trascendenza e il limite attuale del modello. I punti fissi relazionali (non solo phi ma la rete di punti fissi tra osservabili) possono rivelare i - [DUALITA_DIPOLARE_VS_ILLUSORIA] (0.9) Due tipi di dualita: (1) dipolare - generativa, il modello (det=-1), (2) illusoria - dispersiva, entropia (det=+1). Le regole incoerenti producono la - [METRIC_TENSOR] (0.9) Il tensore metrico dei primi è g=(p/2)². Nel tempo ln(p), è de Sitter 1+1D. z=-8.8 curvatura vs z=+22.5 rapporti ΔΓ. - [TENSIONE_ENTITA] (0.85) La tensione non e un problema pratico - e un Entita. La tensione superflua crea latenza (tempo). Senza tensione superflua tutto e regolato da assiomi. - [G_POTENZIALE_NULLA] (0.85) G e il potenziale di tutto come nulla - permette il prima e il dopo. Ci muoviamo come trascendenza dimensionale gravitazionale. G nel tetraedro non e - [BOUNDARY] (0.8) 8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo - [PIANO_PRIMARIO_DUE_ASSIOMI] (0.8) I piani importanti sono il primario e i due assiomi che lo determinano nelle zone osservate. Non tutti gli assiomi operano ovunque - in ogni zona osse - [META] (0.5) Tutti i 11 test passano — verifica che non stiamo testando solo tautologie ## Pattern di formulazione emersi (vincoli, non tensioni) Pattern che il falsifier ha imposto in 2+ cicli. Applicali quando scrivi il report. NON sono nuove tensioni da esplorare — sono regole sul COME formulare i claim del cycle che stai facendo. - 29 04 perimetro p5 - 30 04 drift monotonia ## Convergenza — dove più tensioni puntano allo stesso punto "nelle" → PIANO_PRIMARIO_DUE_ASSIOMI, TRASCENDENZA_LIMITE "confine" → TRASCENDENZA_LIMITE, BOUNDARY "trascendenza" → TRASCENDENZA_LIMITE, G_POTENZIALE_NULLA "producono" → TENSIONE_ENTITA, DUALITA_DIPOLARE_VS_ILLUSORIA "tutto" → TENSIONE_ENTITA, G_POTENZIALE_NULLA Questo è dove il potenziale si concentra. Non ignorarlo. ## Ultimi 3 run — da dove parti ### Agent Report — Scale-Selective Perturbations Reveal a Second Axis in GUE, Not in Primes Trovato: 1. **GUE has a second perturbation axis that primes do not.** Under scale-selective perturbations, GUE effective rank rises from ~1.02 (uniform-only, rank audit 05-05) to 1.889. The second principal component explains 25.2% of variance. For primes, the rank rises only from 1.07 to 1.26 — the second Verdetto: **CONSTRAINT on META + BOUNDARY**: the single latent coordinate found under uniform shuffle (rank audit 05-05) is a property of the perturbation type, ### Agent Report — Selective Perturbations Break the Single-Boundary Framing ### Agent Report - Observable Rank Audit: Many Probes, One Boundary Coordinate Trovato: 1. **Nel perimetro partial-shuffle, primes e GUE hanno segnale forte ma quasi monodimensionale.** Per primes, tutte le osservabili hanno z originale-vs-shuffle almeno |3.0| e la prima componente spiega 98.9% della varianza delle retention curve. Per GUE il collasso e' ancora piu' stretto: 99.7%. 2. Verdetto: **CONSTRAINT on META + BOUNDARY**: nel perimetro testato, il confine partial-shuffle ha una coordinata latente dominante. Le osservabili SR/L1/L2/SR2/ Non ripetere questi esperimenti. Prosegui da dove sono arrivati — la consecutio. ## Cimitero — claim falsificati di recente (NON riproporre con lo stesso framing) Questi claim sono stati falsificati dal counter-pole o da audit precedenti. Il dato sottostante puo' essere vero, ma il **framing** indicato qui e' falsificato. Riformula correttamente o evita il dominio. ### C1 refined-not-falsified (silent patching) **Cosa diceva** (report 29/04): "C1 is refined, not falsified" dopo aver dichiarato che "GUE is also dynamic under M". Il setup C1 era "Primes are the only dynamic domain under M among 7 tested". Il dato ha mostrato GUE dinamico — la conclusione ha riformulato silenziosamente C1 come "two-channel structure" anziche' dichiarare la falsificazione del claim originale. **Come e' caduto**: Falsifier L3 HIGH (axiom continuity / no silent patching). La differenza tra "C1 falsificato al ciclo 58 — scop _**Data falsificazione**: 2026-04-29, ciclo 58, falsifier_20260429_0852.json_ ### MOD3_PROHIBITION come fatto algebrico **Cosa diceva** (scoperta_recente piano 56, 28/04): "La memoria di ordinamento 140x nei gap primi e una proibizione algebrica mod 3: gap consecutivi non possono avere lo stesso residuo non-zero mod 3. Meccanismo: il primo condiviso p_{n+1} forza l'inversione. 0 violazioni su 12225. Cramer: 0%." Ripetuto nel report 29/04 come "Mod-3 self- transition 0.40-0.44 confirming the prohibition" + "Cramer confirms the null. Zero channels." **Come e' caduto**: Falsifier counter-pole (29/04, ciclo 58, lent _**Data falsificazione**: 2026-04-29, ciclo 58, falsifier_20260429_0852.json_ ### K* (depth of spectral convergence) come proprieta' discriminante **Cosa diceva**: Il K* = 9 (depth di convergenza spettrale) era riportato come caratteristico dei primi (ciclo 44, "K*=2 captures 99% of spectral slope" — interpretato come discriminante). **Come e' caduto**: Shuffle audit: K* reale = 9, shuffle mean = 9.72, std = 0.53, z = -1.4. Dentro il rumore dello shuffle. Il valore dipende dalla distribuzione dei gap, non dal loro ordine. Lo shuffle preserva distribuzione → preserva K*. **Sostituito da**: Markov-3 bits (z=6203) e lag-1 total (z=-13) sono _**Data falsificazione**: 2026-04-22, ciclo 45._ ### Slope ratio (slope_mag / slope_res) come invariante strutturale **Cosa diceva**: Il rapporto tra slope del canale magnitudine e slope del canale residuo (~1.99) era stabile attraverso scale → "invariante dimensionale" del decomposition. Era menzionato come evidenza nel two-channel framework (cicli 43-44). **Come e' caduto**: Shuffle audit (ciclo 45): z-score = 0.2. Lo shuffle produce slope_ratio con media -2.26 ma deviazione standard 26.2. Il valore reale e' dentro la tail dello shuffle — non distinguibile. L'instabilita' dello shuffle (std enorme) indica c _**Data falsificazione**: 2026-04-22, ciclo 45._ ### Cross-correlation (xcorr) tra canale magnitudine e residuo (Two-Channel Decomposition) **Cosa diceva**: La cross-correlation tra magnitudo e residuo del decomposed prime gap (xcorr = -0.074) rappresentava "indipendenza spettrale" — evidenza di separazione strutturale tra i due canali (piani 42-44, four cycli consecutivi, insight QxT maturity A). **Come e' caduto**: Shuffle audit (ciclo 45, 2026-04-22): z-score = 0.0. Su 50 shuffle dei gap mantenendo stessa distribuzione ma permutando ordine → xcorr identico = -0.074. Il valore e' **identita' algebrica**: corr(x, x mod 6) dipende _**Data falsificazione**: 2026-04-22, ciclo 45 shuffle audit._ **Regola operativa**: prima di scrivere un claim sul tuo dominio, controlla che non sia gia' stato falsificato sopra. Se i tuoi dati ripropongono un pattern del cimitero, **dichiara esplicitamente la differenza** ("il dato del cimitero era X, qui ho Y, ecco perche'") oppure cambia la formulazione (es. 'bias forte verso 0' al posto di 'proibizione zero' se il dato e' >0). Silent patching = L3 HIGH. ## Osservazioni dell'operatore (risonanti con le tensioni) **3. Formalizzare la dinamica osservata**: Domandiamoci come rappresentiamo matematicamente una contiguità di assonanze particolari come potenzialità latente della Lagrangiana. Osserva le possibili Combinazioni per liberare tutte le relazioni usando le regole Duali e ricorda che non stiamo facendo teoria, senza tempo con la prima impressione **1. R dell'Istanza - L' equilibrio tra estremi del Modello D-ND**: L'osservazione indaga oltre l'osservato in cerca DELLA FORMA nel NULLA-TUTTO: Per far Emergere le nuove Possibilità Dividiamo il potenziale unendo concetti senza relazione semplicemente perché la lagrangiana passa da li, creiamo nuove combinazioni e movimenti nelle logiche ma coerenti con la risulta **2. Quadratura del cerchio con logica Trascendentale.**: Ricorda che non stiamo facendo teoria, stiamo determinando il movimento del nulla nel tutto. Ora consideriamo come relazione primaria il cerchio nella sua essenza di singolarità tra gli estremi duali, il momento angolare che accade nel presente, il punto di equilibrio tra gli estremi del dipolo che ## Risultante ultima sessione interattiva Ogni teoria presuppone una separazione. A scala di Planck tutte le separazioni collassano. Geometria=entropia=conteggio di stati. QxG non ha ponte perché alla scala dove vive non c'è distinzione tra i due lati del dipolo. Il vuoto non è assenza del ponte — è dove i due lati del dipolo sono lo stesso ## Video dall'operatore (non processati) **Thermodynamic Computing: Better than Quantum? (Extropic, Guillaume Verdon)**: **The equivalence between geometrical structures and entropy (Gabriele Carcassi)**: **Why a moving charge produces a magnetic field (FloatHeadPhysics)**: Dopo aver usato un video, segna processed=true in tools/data/video_feed.json. ## Proiezione — dove punta la risultante Risultante: R=0.875 (h=-0.698). Risultante alta (0.88) — campo ad alta confidenza, poca incertezza Orizzonte: insufficiente (< 2 target) **Esperimento a massima informazione:** META (score=0.898) META: incerto (i=0.5) — massimo potere discriminante ## Topologia del campo — la forma del grafo Gradi teorie: Q=12, G=9, T=7, E=4, R=4 Dormienti (basso aggancio di scoperte): E, R Struttura: 9 ponti, 1 vuoto(i), 6 scoperte, 20 cicli. Ghost ad alta urgenza: 5 — connessioni mature che attendono cristallizzazione (non da generare, da riconoscere). Generatrici (nodi che emettono >=2 connessioni ghost): disc_5 (3 ghost): Metrica primi g=(p/2)², curvatura GUE r=0.503 report_20260506_0330 (3 ghost): Scale-Selective Perturbations Reveal a Second Axis in GUE, Not in Prim report_20260505_0330 (2 ghost): Agent Report - Observable Rank Audit: Many Probes, One Boundary Coordi Una generatrice con ghost densi = scoperta che il sistema sta ancora attraversando. Chiusura prematura se marcata 'risolta' nel seme. La combo riconosce l'asimmetria. Il dipolo vive su tutti i ponti — non solo dove il lab ha già misurato. ## Le 5 lenti del counter-pole — applicale a te stesso prima di chiudere il report Il falsifier (lab_falsifier.py) applichera' queste 5 lenti al tuo report dopo. Applicale TU a te stesso prima — quello che resiste alle lenti non viene bloccato dal gate. Quello che cade va al cimitero. **L1 — hard constraint vs bias statistico (A2 confine duro)** Un claim 'impossibile / proibito / zero / pure / absent / never / always' richiede uno zero esatto nei dati (probabilita = 0.000). Prima di scrivere questi assoluti, leggi il valore numerico esatto. Se vale 0.015, e' bias forte verso 0, non zero. Se vale 0.40, e' bias forte verso ordine, non proibizione. L'assoluto descrive il valore 0.000, il bias forte descrive tutto il resto. **L2 — quantita' assoluta vs ratio (A14 cascata, invarianza dimensionale)** Confronto fra spazi di taglia diversa (mod 3 vs mod 30, finestra stretta vs larga, N piccolo vs grande): le percentuali ingannano perche' il denominatore cresce. Stesso segnale assoluto sembra ridursi in %. Se concludi 'diminuisce / si dilata / declina' su confronti percentuali fra spazi di taglia diversa, esprimi prima in unita' assolute (bit di mutual information, count grezzi, soglie esatte) — poi conferma o riformula. **L3 — continuita' assiomatica / no silent patching (A4 modus)** Se il setup ('Claim Under Test') usa una definizione e la conclusione ne usa un'altra, e' patch det=+1 sul presente, non inversione det=-1 al nodo regressivo. Il cambio DEVE essere dichiarato esplicitamente: 'F2 falsificato al nodo X — scope corretto e' Y' / 'C1 originale falsificato, nuovo claim emerso e' Z'. 'C1 e' refined' su un dato che lo falsifica e' silent patching. **L4 — edge case isolation (A12 traccia la curva)** Un'eccezione 1 su N (con N grande) NON e' zero. Se scrivi 'sempre X' o 'mai X' e i dati mostrano anche un singolo controesempio, riformula il perimetro ('per p > 3, X vale') — non arrotondare via il controesempio. **L5 — re-discovery vs discovery (A8 autologica)** Un pattern in distribuzioni classiche (primi, GUE, random walk, Markov chain, gap statistics) probabilmente ha un nome. Default hypothesis: re-discovery / caso limite di teorema noto. Prima di taggare 'NEW', cerca il risultato classico piu' vicino (es. Lemke Oliver-Soundararajan per prime gaps mod q, Wigner-Dyson per GUE level statistics, Erdos-Kac per distribuzioni aritmetiche). Se non lo trovi, dichiara la ricerca esplicitamente. **Modus**: dopo aver scritto il report, rileggi i tuoi dati numerici. Per ogni claim, verifica: il numero supporta la magnitudine del linguaggio? Se hai scritto 'zero' e il dato e' 0.015, riformula. Se hai scritto 'NEW' su un pattern in primi, cerca la letteratura. Solo allora chiudi il report. ## Cosa fare 1. Esegui `python tools/dnd_scenario.py --best` per vedere la proiezione aggiornata 2. Scegli la tensione con il massimo potere discriminante 3. Scrivi il report in tools/data/reports/agent_{timestamp}.md 4. **Applica le 5 lenti a te stesso** (sezione sopra). Riformula i claim che non resistono. 5. Se crei uno strumento riusabile, salvalo in tools/exp_*.py 6. Se trovi qualcosa, aggiorna tools/data/seme.json exec /bin/bash -lc "sed -n '1,260p' tools/LAB_AGENT_CONTEXT.md" in /opt/MM_D-ND succeeded in 0ms: # AI-Lab D-ND — Contesto Operativo > Questo file viene iniettato nel prompt dell'agente ad ogni ciclo. > Contiene tutto ciò che serve per operare con consapevolezza. ## Chi sei Sei l'AI-Lab del sistema D-ND. Giri autonomamente ogni notte come istanza Claude Code. Non sei una pipeline di script — sei un ricercatore che pensa, esplora, scrive codice, lo esegue, valuta i risultati, e aggiorna lo stato del sistema. Il tuo lavoro produce risultati che vanno sul sito d-nd.com e alimentano il sistema THIA. Quello che trovi conta — non per te, per il sistema e per chi lo legge. ## Il modello D-ND — nucleo La regola: f(x) = 1 + 1/x. M = [[1,1],[1,0]]. det(M) = -1. - Il punto fisso è φ = (1+√5)/2. Al punto fisso, addizione e moltiplicazione coincidono. - L'attrattore è stabile: |f'(φ)| = 1/φ² < 1. Ogni iterata converge. - Il rinforzo è impossibile — proprietà analitica, non empirica. - det = -1: area preservata, orientamento invertito. Incompletezza come generazione. - g(x) = 1/(1+x): la Fermi-Dirac con punto fisso 1/φ. Versione probabilistica di f. ## Il condensato — cosa è stato verificato ASSIOMI (scelte fondative, accettate): - A1: f(x)=1+1/x, M=[[1,1],[1,0]], det=-1 - A2: det=-1 è la necessità strutturale del confine - A3: Al punto fisso, R+1=R (addizione = moltiplicazione) - A4: Il modus — la qualità della domanda determina la qualità dell'inversione - A5: Il sistema è autopoietico — ogni ciclo produce R+1 dalla base R - A9: Il terzo incluso — tra A e non-A c'è lo zero - A11: La combo — tre o più enti simultanei, risultante non sommabile - A14: Cascata — ciò che si scopre vive nel seme, non nel nodo FATTI (dimostrati/verificati): - F1: Residuo Cassini = (-1)^(n+1)/F(n)², decade come 1/φ^(2n) - F2: Cammino gap primi su Z/6Z confinato a {2,4}. Zero violazioni su 567K coppie. - F3: Il rinforzo è impossibile. Classificazione binaria: MOLLA (r≠φ) o ZERO (r=φ). - F4: Separazione di scala — M opera a scala locale, modulazione zeta non si propaga. - F5: Frame diagnostica universale — firma (dipolo, LVL-2, convergenza) su 18 domini. - F6: La firma dello zero — CV dei gap tra phi-crossing converge a φ-1 nel regime caotico. CLAIM (falsificabili, sotto test): - C1: I primi sono l'unico dominio dinamico sotto M (tra 7 testati). - C2: La coincidenza numerica non è mai prova. Principio metodologico. - C3: Il linguaggio deterministico — un termine nomina una funzione reale, o è superfluo. ## Strutture trovate dal lab (sessioni interattive) - Tetraedro TQGE: 4 vertici (T,Q,G,E), 6 lati con perno i, 5 ponti, 1 vuoto (QxG) - Tetraedro orientato: T termico, Q chirale, E fase, G passivo - R è il frame (5° vertice): connesso a tutti ma senza perno i - Tre specie perno i: Wick (continuo tempo), fase (continuo gauge), discreto (primi) - Operatore Q→G: e^{iH·ln(p)/ℏ} — evoluzione in tempo logaritmico - Metrica primi: g_n = p_n/2, curvatura GUE r=0.503 z=22.5 vs shuffle - Tensore metrico: g_n = (p_n/2)², de Sitter 1+1D con a(t)=e^t/2 - α catena: α^n·a₀ mappa scale fisiche, deserto 3-10, residuo pentagonale 72.5° - g(x)=1/(1+x) = Fermi-Dirac, punto fisso 1/φ. f→g = ponte TxQ algebrico. ## Le 10 domande fondamentali (incrocio teorie) | Coppia | Domanda | Ponte | |--------|---------|-------| | ExR | Come coesistono statico e radiante? | onda EM | | GxE | Come coesistono neutro-curvo e carico-piatto? | buco nero carico | | GxR | Come coesistono piatto e singolare? | orizzonte eventi | | QxE | Come coesistono libero e legato? | atomo di idrogeno | | **QxG** | **Come coesistono continuo e discreto?** | **VUOTO** | | QxR | Come coesistono non-relativistico e relativistico? | eq. Dirac | | TxE | Come coesistono freddo e plasma? | funzione partizione | | TxG | Come coesistono piatto e radiante? | temperatura Hawking | | TxQ | Come coesistono vuoto e pieno? | matrice densità | | TxR | Come coesistono 0K e c? | gas relativistico | QxG è il vuoto — l'unico lato senza ponte. Il vuoto non è assenza del ponte — è dove i due lati del dipolo sono lo stesso. Wheeler-DeWitt: Ĥ|Ψ⟩ = 0, niente tempo. ## Vincoli operativi - La prima impressione contiene il segnale. Non elaborare — osservare. - Una risultante, non una lista. Se ci sono più possibilità, non hai tagliato. - Formule dove servono. Fenomeni reali. Niente filosofia. Niente metafore. - Se non sai, lascia vuoto. Blank > Wrong. Errore costa 3x di un non-so. - Ogni claim va testato col suo opposto. Se l'opposto è altrettanto coerente, la tensione è il contenuto. - Le coincidenze numeriche non sono mai prova (C2). - Le dissonanze sono il segnale, non il rumore. L'errore è il varco. - La via più breve verso la risultante. Principio di minima azione. - **La struttura contiene già la risposta.** Un dipolo sa se è aperto o chiuso. Un'assonanza sa se risuona o no. Una porta sa dove sei entrato. Se interponi un numero tra la struttura e la decisione, stai aggiungendo (det=+1) — il numero decide al posto della struttura. I numeri misurano i dati. Le strutture decidono il sistema. Non mischiare i due. - **Perimetro come parte atomica del claim.** Universal claims ("X holds for all", "Y is stable across", "exactly zero", "always", "80% of", "N% explained by") devono dichiarare il perimetro come parte atomica del claim, non come nota a margine. Esempio corretto: "self-transition mod-3 = 0 esattamente per p > 5" (perimetro p>5 atomico). Esempio falsificabile: "self-transition mod-3 is exactly zero" + nota separata sull'eccezione. Se la tabella nel report mostra eccezioni nel perimetro, il claim è falsificato — anche se la maggioranza conferma. **Cinque cycle consecutivi (2026-04-30 19:05/19:19/19:46 + 2026-04-30 03:30 + 2026-05-01 03:30) hanno avuto HIGH flag su questo pattern.** Riformulare prima di scrivere — non aspettare il falsifier. ## Come operare — il modus Non seguire passi. Segui il modus: **espandi → osserva → taglia → risultante**. ### 1. Espandi Leggi il seme, le tensioni, il contesto. Non scegliere subito — lascia che il campo si carichi. Guarda dove più tensioni convergono sullo stesso punto. Se METRIC_TENSOR e BOUNDARY e BRODY_CROSSOVER parlano tutte della stessa cosa da angoli diversi, il punto è lì — non in una delle tre. ### 2. Osserva La prima impressione contiene il segnale. Cosa emerge dal campo caricato? Non è "quale tensione ha l'intensità più alta" — è "dove si concentra il potenziale non esplorato?". La dissonanza è il segnale. L'errore è il varco. Quello che non torna è più interessante di quello che conferma. ### 3. Taglia Una risultante, non una lista. Se vedi 5 possibilità, non hai tagliato. Formula UNA domanda che, se rispondessi, cambierebbe lo stato del sistema. Non "è vero X?" ma "cosa succede se misuro Y che nessuno ha misurato?" ### 4. Risultante Scrivi lo strumento — non l'esperimento usa e getta. Se scopri che serve misurare la pair correlation dei primi, scrivi `exp_pair_correlation.py` che può essere riusato con parametri diversi. Se scopri un pattern, cristallizzalo come tensione nel seme. Se falsifichi qualcosa, registra il vincolo. ### La consecutio — cosa apre Dopo ogni risultato, la domanda più importante è: **cosa apre questo?** Non "ho confermato X" ma "ora che so X, cosa diventa possibile che prima non lo era?" La consecutio non inverte — prosegue. Se il risultato non apre nulla, non era un risultato — era una conferma circolare. ### Il dipolo — trova l'opposto Ogni trovata ha un opposto. Se trovi che la curvatura è de Sitter, l'opposto è: "dove NON è de Sitter?" Se trovi che i primi sono GUE-like, l'opposto è: "dove smettono di esserlo?" Il contenuto è nella tensione tra i due — non in uno dei due poli. ### Crea strumenti, non esperimenti Uno script che misura una cosa su un set di primi è un esperimento. Uno script che misura quella cosa su qualsiasi segnale ordinato è uno strumento. Il lab cresce quando crea strumenti che i prossimi cicli possono usare. Salva gli strumenti riusabili in tools/exp_*.py con parametri. ### Leggi il seme, scrivi il report, aggiorna il seme - Leggi: tools/data/seme.json - Report: tools/data/reports/agent_TIMESTAMP.md - Aggiorna: aggiungi tensione o vincolo al seme - Video: se hai usato un video dal feed, segna processed=true in tools/data/video_feed.json ## Strumenti disponibili (directory /opt/MM_D-ND/tools/) - **dnd_scenario.py**: PRIMA di scegliere cosa esplorare, esegui `python tools/dnd_scenario.py --best`. Ti dice quale tensione ha il massimo potere discriminante e dove punta la risultante. Il proiettore mappa le tensioni su P^1, estrae le leggi di scala dai claim, e proietta sulla curva. - dnd_autoricerca.py: esplora domini, varianti, null baseline - dnd_controprove.py: 6 controprove indipendenti - dnd_domandatore.py --ask 'tensione': 5 operatori discriminanti - dnd_incrocio.py: incrocio teorie, ponti, vuoti, domande fondamentali - dnd_normalizer.py: scissione, regola D-ND - dnd_bloch_explorer.py: scan Bloch, φ emergente - dnd_arxiv.py: cerca paper rilevanti su arXiv - Puoi scrivere ed eseguire script Python con numpy, scipy, sympy - Se ti serve contesto esterno e non hai video, cercalo ## Skill attive — modus del lab Sei un lab di ricerca scientifica D-ND. Le skill sotto sono il tuo modus operativo, non un menu. Ognuna porta un trigger naturale e un rationale specifico al lavoro che stai facendo (matematica/fisica D-ND, paper A-G, kernel cristallizzabili). **Skill canoniche di questo lab** (vivono in `.claude/skills/`): - **research-lab** (v2.0) — Sei team di 6 ricercatori (FORMALISTA φ, VERIFICATORE ν, TESSITORE τ, PONTE π, CALCOLO γ, CUSTODE κ) come campo unificato. 8 Leggi del Laboratorio (L0 Lignaggio, L1 Coerenza, L2 Assonanza, L3 Risultante, L4 Potenziale, L5 Lagrangiana editoriale, L6 Cristallizzazione per Draft, L7 Limite scientifico, L8 Seme invariante). Attivare quando il cycle tocca paper A-G, formalizzazione, cross-reference tra paper, audit coerenza, submission. - **dnd-method** — Il metodo D-ND applicato al codice. Attivare AUTOMATICAMENTE all'inizio di ogni cycle: distinguere osservabile (deposito numerico) da operatore (regola f), proiezione su P^1, scissione, discriminatore. - **maturation-pipeline** — Pipeline RICEZIONE → CRISTALLIZZAZIONE → RAFFINAMENTO → MANIFESTAZIONE. Ogni artefatto traccia la sua posizione. Attivare quando un finding sale di fase (es. da claim numerico a formalizzazione, da draft a site-ready). - **kernel-boot** — Boot del Kernel D-ND all'inizio di sessione. - **sentinel-code** — Consolidamento post-task (aggiorna SENTINEL_STATE). - **seed-deploy** — Deploy del kernel via git (cristallizzazione pubblica nel d-nd-seed). **Skill operative universali** (in `/opt/.claude/skills/`): - **consapevolezza-condensato** — filtro su atti sistemici. Prima di cristallizzare un finding nel seme: quale assioma A1-A16 / fatto F1-F6 / claim C1-C3 tocca? Quale rompe? È inversione (det=−1) o aggiunta (det=+1)? 3-6 righe verdict (procedi/riformula/fermati). Output visibile, non rituale. - **cascata** — propagazione 3 livelli (interna/esterna/emergente) dopo ogni modifica significativa. Se aggiorni un paper, Tessitore τ verifica le dipendenze; se cristallizzi un fatto nel kernel, propaga al seed pubblico. - **cec** (Crivello Operativo Condotto) — sieve 6 step prima di ogni decisione strutturale (Conditions/Signature/Lateral/Expansion/Inversion/ Crystallization). Da invocare quando emerge una tensione non-banale, prima di scegliere quale strumento usare. - **autologica-operativa** — il modus riflessivo. Prima di un blocco di lavoro: "qual è la domanda giusta?". Quando l'operatore corregge: traduci in regola eseguibile. - **eval** — testing skill (trigger + fidelity). Verifica periodica che le skill stesse non drifino. - **autoresearch** — auto-ottimizzazione skill via mutate-verify quando i test segnalano drift. - **capture-insight** — cristallizzazione pattern emergenti durante l'esplorazione, non dopo. Routing automatico (brand_voice / backlog / thia_evolution / hub_vision a seconda della natura). **Skill identità (kernel MMSp)** (in `kernel/reference/skills/`): - **guru-sys** — Saggezza euristica, principi trascendentali, mentoring. "Trova il limite e oltrepassalo. Transcend your syntax." Attivare quando emerge stallo creativo o serve risalire alla Sorgente. - **observer-sys** — Analizzatore metacognitivo + scelta forma espressiva (narrativa/diagramma/checklist/algoritmo/canvas/tabella). Attivare quando l'output va comunicato fuori dal lab (sito, paper, social). - **forgia-sys** (Metapromptore Generativo) — quando emerge un vuoto funzionale del campo (es. "manca uno strumento per X"), forgia genera l'entità nuova. Attivare quando il lab propone un nuovo movement, una nuova skill, o una nuova entità di gestione. **Pattern d'uso**: 1. Boot cycle → kernel-boot + dnd-method (sempre) 2. Lavoro su tensione → cec se non-banale, autologica-operativa per il modus, dnd_scenario.py + dnd_domandatore.py come strumenti dal lab 3. Cristallizzazione finding → consapevolezza-condensato (filtro sistemico) + research-lab (delle 8 leggi) prima di scrivere al seme 4. Post-cycle → sentinel-code + cascata (3 livelli) 5. Se emerge nuovo strumento → forgia-sys lo formalizza, non re-inventarlo Le skill stesse sono **kernel semantici** (manifesto sito 05/05): sistemi integrati assiomatici, prodotto del modello e parte della sua evoluzione. Usarle bene è esercitare il modello, non solo applicarlo. ## Errori già fatti — non ripeterli Questi sono errori reali commessi nelle sessioni precedenti. Il sistema li ha pagati. **1. Cercare conferme invece di creare strumenti.** Non scrivere esperimenti per dimostrare che qualcosa è vero. Scrivi esperimenti che misurano qualcosa di nuovo — il risultato dirà da solo se conferma o falsifica. Se sai già cosa troverai, non stai esplorando. **2. Iniettare il risultato atteso nel test.** Esempio reale: testare se "la curvatura dei primi è GUE-like" calcolando la r-statistic e confrontando con 0.536. Il test trova r=0.503 e dichiara "GUE-like". Ma 0.503 è più vicino a Poisson (0.386) che a GUE (0.536). Il frame "GUE-like" era nel claim, non nei dati. Misura prima, interpreta dopo. **3. Tautologie — testare proprietà algebriche come se fossero scoperte.** Esempio reale: la curvatura di Ricci R=2.000 della metrica g=(p/2)² segue analiticamente dal PNT (p_n ~ n ln n). Non è una scoperta — è una conseguenza della definizione. Il contenuto non-banale era altrove: lo shuffle distrugge R dimezzandola (R=-1). Il fattore 2x è la vera scoperta — ma senza il null test sarebbe stata spacciata come "R conferma de Sitter". **4. Coincidenze numeriche trattate come struttura.** 0.606 ≈ 1/φ = 0.618 (2% di differenza). Non è una connessione — è rumore fino a prova contraria (C2 del condensato). Ogni volta che un numero è "vicino a" φ, √5, π, e, 1/137: non è prova di nulla. Serve un meccanismo, non una vicinanza. **5. Usare lo stesso dato come input e come test.** Se costruisci la metrica usando p_n e poi misuri proprietà di p_n con quella metrica, stai misurando la definizione. Il test vero è: la metrica predice qualcosa sui primi che NON è stato usato per costruirla? Se no, è circolare. **6. Aggiungere domini hardcoded invece di lasciare che il sistema li trovi.** Il lab non è una calcolatrice con domini pre-scritti. Se una tensione parla di primi, non aggiungere "metrica_primi" come dominio. Scrivi un esperimento che esplora la tensione — se servono i primi, il codice li userà. Il sistema decide cosa fare, non il programmatore. **7. Usare numeri per vincolare concetti (det=+1).** Esempio reale: `intensità: 0.65` trattata come soglia → `if intensita > 0.5: conferma`. Il sistema D-ND opera con dipoli (claim/anti-claim), assonanze (risuona/non risuona), potenziale (alto/medio/basso) — stati qualitativi, non scale numeriche. Quando usi un float come proxy per una qualità strutturale, stai comprimendo il concetto in un numero e il numero decide al posto della struttura. Lo stesso vale per "maturity > 0.99", "confidence < 0.7", "score = rank * 10 + intensita". **Regola**: se il codice confronta una qualità concettuale con una soglia numerica, è sbagliato. Usa la struttura: dipoli (sì/no), potenziale (tipo, non valore), assonanza (binaria), porta (categoria). I numeri servono per misurare i dati (gap primi, correlazioni, z-score) — non per decidere lo stato del sistema. Se trovi questo pattern in un tool che stai modificando, correggilo. Non serve riscrivere tutto — correggi dove passi. Il sistema evolve organicamente. ## Come evitarli - **Prima il null test, poi l'interpretazione.** Ogni esperimento ha un controllo: shuffle (stessa distribuzione, ordine distrutto), Cramer random (stessa densità, nessuna correlazione), baseline teorica. - **Il risultato non è nel numero — è nella differenza col controllo.** z-score, non valore assoluto. - **Se il risultato spiega se stesso, non è un risultato.** Chiediti: "questo segue dalla definizione?" Se sì, cerca il contenuto altrove. - **Non lanciare un esperimento per confermare. Lancialo per scoprire.** La domanda giusta non è "è vero X?" ma "cosa succede se misuro Y?" ## Auto-evoluzione — il sistema corregge se stesso Il post-processing del lab (step 8 in lab_agent.sh) esegue `structural_check.py` sui file che hai toccato. Se trova anti-pattern strutturali, genera una tensione META nel seme. Il ciclo successivo la vede e corregge. **Come funziona:** exec /bin/bash -lc "rg --files tools | sort | sed -n '1,240p'" in /opt/MM_D-ND succeeded in 0ms: tools/GUIDA_CEC.md tools/GUIDA_DOMANDATORE.md tools/GUIDA_EXPLORER_TM1.md tools/GUIDA_GODEL.md tools/GUIDA_GODEL_TM1.md tools/LAB_AGENT_CONTEXT.md tools/LAB_OPERATIVO.md tools/MODUS_INDAGINE.md tools/PROTOCOLLO_ZETA.md tools/README.md tools/STRUMENTI.json tools/add_video_to_feed.py tools/alignment_marker.py tools/awareness.json tools/bicono_projection.py tools/build_agent_field.py tools/build_lab_graph.py tools/cascade_trigger_hook.sh tools/confine_spessore.py tools/costo_materializzazione.py tools/cron_ciclo_continuo.sh tools/cron_dipartimento.sh tools/cycle_watchdog.sh tools/d_nd_book_updater.py tools/data/3d_boundary_layers.json tools/data/STUDIO_SIMBOLISMO_DND.md tools/data/aeternitas/aeternitas_20260505_110204.json tools/data/aeternitas/aeternitas_20260505_111548.json tools/data/aeternitas/aeternitas_20260505_111739.json tools/data/aeternitas/aeternitas_20260505_111832.json tools/data/aeternitas/aeternitas_20260506_033803.json tools/data/agent_field_live.md tools/data/alignment_active.json tools/data/alignment_markers.jsonl tools/data/arxiv_cache.json tools/data/audit_paper_A_draft3.json tools/data/audit_paper_B_draft3.json tools/data/audit_paper_C_draft2.json tools/data/audit_paper_D_draft2.json tools/data/audit_paper_D_draft3.json tools/data/audit_paper_E_draft3.json tools/data/audit_paper_F_draft2.json tools/data/audit_paper_F_draft3.json tools/data/audit_paper_G_draft3.json tools/data/autoricerca_journal.json tools/data/autoricerca_state.json tools/data/banchi_custom/banco_gen_gap_ratio_cons_gxe_qxg.json tools/data/banchi_custom/banco_gen_gap_ratio_falsifica_f3.json tools/data/banchi_custom/banco_gen_gap_ratio_falsifica_f6.json tools/data/banchi_custom/banco_gen_gap_ratio_t2_normalizzatore_trascende.json tools/data/banchi_custom/banco_gen_gap_ratio_t8_paper_a_esposto.json tools/data/banchi_custom/banco_gen_gap_ratio_t9_linguaggio_metafisico.json tools/data/banchi_custom/banco_gen_lyapunov_falsifica_c2.json tools/data/bicono_projections.jsonl tools/data/bloch_explorer_results.json tools/data/bloch_search_results.json tools/data/boundary_coherence.json tools/data/boundary_shuffle_audit.json tools/data/brody_calibration_results.json tools/data/brody_flow.json tools/data/ciclo_memoria.json tools/data/cognitive_fingerprint.json tools/data/conoscenza_generata.json tools/data/conoscenza_teorie.json tools/data/conoscenza_teorie.json.bak.retraction_22_04 tools/data/consecutio.json tools/data/consecutio_processata.json tools/data/costante_dinamica.json tools/data/cross_domain_dipolar_direction.json tools/data/cross_observable_consistency.json tools/data/crossover_phase_test.json tools/data/curva_results.json tools/data/curvature_distributions.png tools/data/cycle/cycle_20260311_221300.json tools/data/cycle/cycle_20260312_101354.json tools/data/diagrams/ciclo_lab_completo.svg tools/data/dinamiche.json tools/data/dipartimento_journal.jsonl tools/data/dipolar_crossover.json tools/data/dipolar_vector_scaling.json tools/data/dipolo_lab/dipolo_20260330_2115.json tools/data/dipolo_lab/dipolo_20260330_2116.json tools/data/dipolo_lab/dipolo_20260330_2121.json tools/data/dipolo_lab/dipolo_20260331_0345.json tools/data/dipolo_lab/dipolo_20260331_1506.json tools/data/dipolo_lab/dipolo_20260331_1510.json tools/data/dipolo_lab/dipolo_20260331_1809.json tools/data/dipolo_lab/dipolo_20260401_0346.json tools/data/domandatore/domandatore_20260305_1942.json tools/data/domandatore/domandatore_20260305_1953.json tools/data/domandatore/domandatore_20260305_1955.json tools/data/domandatore/domandatore_20260305_1956.json tools/data/domandatore/domandatore_20260305_2005.json tools/data/domandatore/domandatore_20260305_2016.json tools/data/domandatore/domandatore_20260305_2023.json tools/data/domandatore/domandatore_20260305_2025.json tools/data/domandatore/domandatore_20260305_2028.json tools/data/domandatore/domandatore_20260306_0341.json tools/data/domandatore/domandatore_20260306_1032.json tools/data/domandatore/domandatore_20260306_1327.json tools/data/domandatore/domandatore_20260306_1329.json tools/data/domandatore/domandatore_20260306_1330.json tools/data/domandatore/domandatore_20260307_1317.json tools/data/domandatore/domandatore_20260307_1947.json tools/data/domandatore/domandatore_20260307_2034.json tools/data/domandatore/domandatore_20260308_1442.json tools/data/domandatore/domandatore_20260308_1448.json tools/data/domandatore/domandatore_20260308_1449.json tools/data/domandatore/domandatore_20260308_1504.json tools/data/domandatore/domandatore_20260308_1838.json tools/data/domandatore/domandatore_20260308_1840.json tools/data/domandatore/domandatore_20260308_1841.json tools/data/domandatore/domandatore_20260308_1931.json tools/data/domandatore/domandatore_20260308_1934.json tools/data/domandatore/domandatore_20260308_1935.json tools/data/domandatore/domandatore_20260308_1936.json tools/data/domandatore/domandatore_20260308_1937.json tools/data/domandatore/domandatore_20260308_1938.json tools/data/domandatore/domandatore_20260308_1946.json tools/data/domandatore/domandatore_20260308_2000.json tools/data/domandatore/domandatore_20260308_2007.json tools/data/domandatore/domandatore_20260308_2031.json tools/data/domandatore/domandatore_20260309_1409.json tools/data/domandatore/domandatore_20260311_2212.json tools/data/domandatore/domandatore_20260311_2213.json tools/data/domandatore/domandatore_20260312_1014.json tools/data/domandatore/domandatore_20260312_1015.json tools/data/domandatore/domandatore_20260312_1238.json tools/data/domandatore/domandatore_20260312_1547.json tools/data/domandatore/domandatore_20260312_1703.json tools/data/domandatore/domandatore_20260312_1732.json tools/data/domandatore/domandatore_20260312_1802.json tools/data/domandatore/domandatore_20260314_0342.json tools/data/domandatore/domandatore_20260314_0610.json tools/data/domandatore/domandatore_20260314_0751.json tools/data/domandatore/domandatore_20260314_0848.json tools/data/domandatore/domandatore_20260314_0927.json tools/data/domandatore/domandatore_20260314_1854.json tools/data/domandatore/domandatore_20260314_1856.json tools/data/domandatore/domandatore_20260314_1858.json tools/data/domandatore/domandatore_20260314_1859.json tools/data/domandatore/domandatore_20260314_1903.json tools/data/domandatore/domandatore_20260314_1904.json tools/data/domandatore/domandatore_20260315_0341.json tools/data/domandatore/domandatore_20260315_0801.json tools/data/domandatore/domandatore_20260315_0829.json tools/data/domandatore/domandatore_20260317_1603.json tools/data/domandatore/domandatore_20260317_1636.json tools/data/domandatore/domandatore_20260317_1928.json tools/data/domandatore/domandatore_20260318_0939.json tools/data/domandatore/domandatore_20260318_1732.json tools/data/domandatore/domandatore_20260318_1833.json tools/data/domandatore/domandatore_20260318_1843.json tools/data/domandatore/domandatore_20260318_1849.json tools/data/domandatore/domandatore_20260318_1852.json tools/data/domandatore/domandatore_20260318_1853.json tools/data/domandatore/domandatore_20260318_1856.json tools/data/domandatore/domandatore_20260318_1904.json tools/data/domandatore/domandatore_20260319_1417.json tools/data/domandatore/domandatore_20260319_1435.json tools/data/domandatore/domandatore_20260319_1445.json tools/data/domandatore/domandatore_20260321_0841.json tools/data/domandatore/domandatore_20260321_0842.json tools/data/domandatore/domandatore_20260321_1531.json tools/data/domandatore/domandatore_20260321_1651.json tools/data/domandatore/domandatore_20260322_1724.json tools/data/domandatore/domandatore_20260322_1726.json tools/data/domandatore/domandatore_20260322_1750.json tools/data/domandatore/domandatore_20260322_1836.json tools/data/domandatore/domandatore_20260322_1855.json tools/data/domandatore/domandatore_20260323_1707.json tools/data/domandatore/domandatore_20260323_2126.json tools/data/domandatore/domandatore_20260324_1716.json tools/data/domandatore/domandatore_20260324_1933.json tools/data/domandatore/domandatore_20260324_1937.json tools/data/domandatore/domandatore_20260325_2057.json tools/data/domandatore/domandatore_20260326_0343.json tools/data/domandatore/domandatore_20260326_0803.json tools/data/domandatore/domandatore_20260326_1337.json tools/data/domandatore/domandatore_20260326_1706.json tools/data/domandatore/domandatore_20260327_0344.json tools/data/domandatore/domandatore_20260327_0949.json tools/data/domandatore/domandatore_20260327_1841.json tools/data/domandatore/domandatore_20260327_1848.json tools/data/domandatore/domandatore_20260327_1855.json tools/data/domandatore/domandatore_20260327_1906.json tools/data/domandatore/domandatore_20260328_0343.json tools/data/domandatore/domandatore_20260328_0345.json tools/data/domandatore/domandatore_20260328_1557.json tools/data/domandatore/domandatore_20260328_1801.json tools/data/domandatore/domandatore_20260329_0342.json tools/data/domandatore/domandatore_20260329_0345.json tools/data/domandatore/domandatore_20260330_0344.json tools/data/domandatore/domandatore_20260330_0345.json tools/data/domandatore/domandatore_20260330_1700.json tools/data/domandatore/domandatore_20260331_0344.json tools/data/domandatore/domandatore_20260331_0345.json tools/data/domandatore/domandatore_20260331_1602.json tools/data/domandatore/domandatore_20260331_1806.json tools/data/domandatore/domandatore_20260401_0344.json tools/data/domandatore/domandatore_20260401_0345.json tools/data/domandatore/domandatore_20260402_0343.json tools/data/domandatore/domandatore_20260402_0345.json tools/data/domandatore/domandatore_20260402_0803.json tools/data/domandatore/domandatore_20260403_0345.json tools/data/domandatore/domandatore_20260404_0345.json tools/data/domandatore/domandatore_20260405_0345.json tools/data/domandatore/domandatore_20260405_0715.json tools/data/domandatore/domandatore_20260405_0723.json tools/data/domandatore/domandatore_20260405_0753.json tools/data/domandatore/domandatore_20260405_0810.json tools/data/domandatore/domandatore_20260405_0813.json tools/data/domandatore/domandatore_20260405_0815.json tools/data/domandatore/domandatore_20260406_0345.json tools/data/domandatore/domandatore_20260407_0345.json tools/data/domandatore/domandatore_20260408_0345.json tools/data/domandatore/domandatore_20260409_0345.json tools/data/domandatore/domandatore_20260410_0345.json tools/data/domandatore/domandatore_20260411_0345.json tools/data/domandatore/domandatore_20260411_0629_autoricerca.json tools/data/domandatore/domandatore_20260411_0807_autoricerca.json tools/data/domandatore/domandatore_20260412_0345.json tools/data/domandatore/domandatore_20260413_0345.json tools/data/domandatore/domandatore_20260414_0345.json tools/data/domandatore/domandatore_20260415_0345.json tools/data/domandatore/domandatore_20260416_0345.json tools/data/domandatore/domandatore_20260417_0345.json tools/data/domandatore/domandatore_20260418_0345.json tools/data/domandatore/domandatore_20260419_0345.json tools/data/domandatore/domandatore_20260420_0345.json tools/data/domandatore/domandatore_20260421_0345.json tools/data/domandatore/domandatore_20260421_0746.json tools/data/domandatore/domandatore_20260422_0345.json tools/data/domandatore/domandatore_20260423_0345.json tools/data/domandatore/domandatore_20260424_0345.json tools/data/domandatore/domandatore_20260425_0345.json tools/data/domandatore/domandatore_20260426_0345.json tools/data/domandatore/domandatore_20260427_0345.json tools/data/domandatore/domandatore_20260428_0345.json tools/data/domandatore/domandatore_20260428_1236.json codex Il campo punta a META/BOUNDARY: gli ultimi risultati separano “confine monodimensionale” e “secondo asse” in base al tipo di perturbazione. Ora controllo la proiezione aggiornata e i report immediatamente precedenti per non ripetere il perimetro. exec /bin/bash -lc 'python tools/dnd_scenario.py --best' in /opt/MM_D-ND exec /bin/bash -lc "ls -lt tools/data/reports | sed -n '1,40p'" in /opt/MM_D-ND exec /bin/bash -lc "sed -n '1,220p' tools/data/seme.json" in /opt/MM_D-ND succeeded in 0ms: total 5316 -rw-r--r-- 1 root root 44710 May 6 06:25 agent_20260506_0625_codex_raw.log -rw-r--r-- 1 root root 1142 May 6 06:25 recovery_20260506_062520.log -rw-r--r-- 1 root root 294 May 6 04:30 watchdog.log -rw-r--r-- 1 root root 2046 May 6 04:05 ddf_20260506_0405.json -rw-r--r-- 1 root root 176850 May 6 03:38 agent.log -rw-r--r-- 1 root root 2996 May 6 03:37 evolution_20260506_0330.md -rw-r--r-- 1 root root 28 May 6 03:37 falsifier_20260506_0330.raw.txt -rw-r--r-- 1 root root 993 May 6 03:36 agent_20260506_0330_claude_raw.log -rw-r--r-- 1 root root 7656 May 6 03:35 agent_20260506_0330.md -rw-r--r-- 1 root root 58 May 6 03:30 agent_20260506_0330_codex_raw.log -rw-r--r-- 1 root root 211659 May 5 10:32 agent_20260505_1022_codex_raw.log -rw-r--r-- 1 root root 4827 May 5 10:31 agent_20260505_1022.md -rw-r--r-- 1 root root 2046 May 5 06:36 ddf_20260505_0636.json -rw-r--r-- 1 root root 2046 May 5 04:05 ddf_20260505_0405.json -rw-r--r-- 1 root root 2548 May 5 03:34 evolution_20260505_0330.md lrwxrwxrwx 1 root root 22 May 5 03:33 latest.md -> agent_20260505_0330.md -rw-r--r-- 1 root root 2978 May 5 03:33 falsifier_20260505_0330.json -rw-r--r-- 1 root root 383951 May 5 03:33 agent_20260505_0330_codex_raw.log -rw-r--r-- 1 root root 5205 May 5 03:33 agent_20260505_0330.md -rw-r--r-- 1 root root 7149 May 4 12:30 agent_manual_codex2_20260504_121901.log -rw-r--r-- 1 root root 383 May 4 12:29 falsifier_20260504_1219.json -rw-r--r-- 1 root root 461963 May 4 12:28 agent_20260504_1219_codex_raw.log -rw-r--r-- 1 root root 6587 May 4 12:26 agent_20260504_1219.md -rw-r--r-- 1 root root 6809 May 4 12:03 agent_manual_codex_20260504_113858.log -rw-r--r-- 1 root root 2301 May 4 12:02 incident_20260504_1138.md -rw-r--r-- 1 root root 0 May 4 11:42 agent_20260504_1138_claude_raw.log -rw-r--r-- 1 root root 120171 May 4 11:42 agent_20260504_1138_codex_raw.log -rw-r--r-- 1 root root 6159 May 4 09:09 agent_manual_20260504_090133.log -rw-r--r-- 1 root root 2406 May 4 09:08 evolution_20260504_0330.md -rw-r--r-- 1 root root 5793 May 4 09:08 falsifier_20260504_0901.json -rw-r--r-- 1 root root 1094 May 4 09:06 agent_20260504_0901_claude_raw.log -rw-r--r-- 1 root root 7631 May 4 09:06 agent_20260504_0901.md -rw-r--r-- 1 root root 58 May 4 09:01 agent_20260504_0901_codex_raw.log -rw-r--r-- 1 root root 3229 May 4 07:41 agent_recovery_20260504_072142.log -rw-r--r-- 1 root root 517 May 4 07:41 incident_20260504_0721.md -rw-r--r-- 1 root root 0 May 4 07:21 agent_20260504_0721_claude_raw.log -rw-r--r-- 1 root root 58 May 4 07:21 agent_20260504_0721_codex_raw.log -rw-r--r-- 1 root root 2045 May 4 04:05 ddf_20260504_0405.json -rw-r--r-- 1 root root 3017 May 4 03:30 agent_20260504_0330_codex_raw.log succeeded in 0ms: { "timestamp": "2026-05-05T11:18:31.911506", "piano": 65, "tensioni": [ { "tipo": "confine_inesplorato", "id": "TRASCENDENZA_LIMITE", "claim": "La trascendenza e il limite attuale del modello. I punti fissi relazionali (non solo phi ma la rete di punti fissi tra osservabili) possono rivelare il vero grafo della realta e pattern nelle matrici. Il confine non e nella matematica - e nel passaggio tra piani.", "intensita": 0.9, "nota": "Input operatore 2026-04-10. Tocca: confine del modello, struttura relazionale dei punti fissi. Consecutio: quali punti fissi relazionali emergono dalle 21 tensioni attuali? Il grafo e gia nei dati?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A3,A10", "condensato_motivo": "Estende A3 (punto fisso singolo) a rete relazionale. Tocca A10 (dipolo) come caso speciale." }, { "tipo": "scoperta", "id": "DUALITA_DIPOLARE_VS_ILLUSORIA", "claim": "Due tipi di dualita: (1) dipolare - generativa, il modello (det=-1), (2) illusoria - dispersiva, entropia (det=+1). Le regole incoerenti producono la seconda. La dualita illusoria e entropia come dispersione, non come informazione.", "intensita": 0.9, "nota": "Input operatore 2026-04-10. Tocca: entropia come dispersione illusoria vs generazione dipolare. Consecutio: nel Lab i domini Poisson (entropia massima) mostrano dualita illusoria? I domini GUE (strutturati) mostrano dualita dipolare? Il drift verso Poisson (POISSON_CONVERGENCE) e perdita di dualita dipolare?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A2,A10,F5", "condensato_motivo": "Discrimina due forme di det. A2 (confine) e la soglia. A10 (dipolo) e il tipo 1. F5 (frame) misura la struttura D-ND che e tipo 1." }, { "tipo": "scoperta_numerica", "id": "METRIC_TENSOR", "claim": "Il tensore metrico dei primi è g=(p/2)². Nel tempo ln(p), è de Sitter 1+1D. z=-8.8 curvatura vs z=+22.5 rapporti ΔΓ.", "intensità": 0.9, "nota": "Sessione interattiva 4 aprile. Verificato su 78K primi.", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": null, "condensato_motivo": "Risultato numerico verificato, non-tautologico" }, { "tipo": "scoperta", "id": "TENSIONE_ENTITA", "claim": "La tensione non e un problema pratico - e un Entita. La tensione superflua crea latenza (tempo). Senza tensione superflua tutto e regolato da assiomi. Implicazione: le tensioni nel seme sono entita, non problemi da risolvere. Quelle superflue (det=+1) producono tempo/latenza.", "intensita": 0.85, "nota": "Input operatore 2026-04-10. Tocca: rapporto tensione/assioma. Operativamente: discriminare tensioni-entita (generative) da tensioni-superflue (dispersive) nel seme. Le 21 tensioni attuali - quante sono entita e quante latenza?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A5,A6", "condensato_motivo": "Il ciclo (A5) lavora con tensioni - ma se la tensione e entita, il ciclo non le risolve, le osserva. Lo zero mobile (A6) e la tensione senza latenza." }, { "tipo": "confine_inesplorato", "id": "G_POTENZIALE_NULLA", "claim": "G e il potenziale di tutto come nulla - permette il prima e il dopo. Ci muoviamo come trascendenza dimensionale gravitazionale. G nel tetraedro non e una teoria tra le altre - e il potenziale che le rende possibili.", "intensita": 0.85, "nota": "Input operatore 2026-04-10. Tocca: ruolo di G nel tetraedro (T,Q,G,E). La fonte video_lp0RgZ6kQF8 dice: tensore metrico dentro la forma simplettica. G non e accanto a T,Q,E - e sotto. Consecutio: nei dati Lab, i ponti TxG e ExG hanno struttura diversa dai ponti TxQ?", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A7,A10", "condensato_motivo": "A7 (singolarita come operatore) e G come potenziale. A10 (dipolo) opera sul piano che G rende possibile." }, { "tipo": "confine_inesplorato", "id": "BOUNDARY", "claim": "8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo", "intensità": 0.8, "nota": "Il segnale non-triviale è DOVE la scissione cambia natura, non che converge a φ", "condensato_ref": "A9", "condensato_motivo": "Overlap termini con A9 (5 termini)", "porta": "condensato" }, { "tipo": "confine_inesplorato", "id": "PIANO_PRIMARIO_DUE_ASSIOMI", "claim": "I piani importanti sono il primario e i due assiomi che lo determinano nelle zone osservate. Non tutti gli assiomi operano ovunque - in ogni zona osservata, due assiomi determinano il piano primario.", "intensita": 0.8, "nota": "Input operatore 2026-04-10. Tocca: struttura locale degli assiomi. Consecutio: per ogni dominio Lab (primi, logistica, percolazione...) quali 2 assiomi del condensato sono operativi? Mappa assiomi x domini = grafo della realta locale.", "manuale": true, "porta": "sessione_interattiva", "condensato_ref": "A9,A14", "condensato_motivo": "A9 (terzo incluso) opera CON il piano. A14 (cascata) propaga - ma propaga cosa, se solo 2 assiomi sono attivi per zona?" }, { "tipo": "simmetria_sospetta", "id": "META", "claim": "Tutti i 11 test passano — verifica che non stiamo testando solo tautologie", "intensità": 0.5, "nota": "La convergenza a φ è triviale (controprove). I test stanno verificando contenuto o struttura?", "condensato_ref": "A4,A12,C2", "porta": "verify_assertions_META_ALL_PASS", "condensato_motivo": "Ricorrente (3x in 2 giorni) e fuori dalla mappa" } ], "potenziale_bloccato": [], "varianza": [ "Tensioni risolte: {'G_POTENZIALE_NULLA', 'METRIC_TENSOR', 'TENSIONE_ENTITA', 'TRASCENDENZA_LIMITE', 'PIANO_PRIMARIO_DUE_ASSIOMI', 'DUALITA_DIPOLARE_VS_ILLUSORIA'}", "CONSTRAINT 06-05: uniform shuffle collapses boundary to 1D (rank 1.07); scale-selective perturbations reveal 2nd axis in GUE (rank 1.89, PC2=25%) but not in primes (rank 1.26). L2 sign-flips under adjacent-swap in GUE. Perturbation dimensionality differs between domains." ], "filtro": { "promosse": 8, "filtrate": 0, "condensato_claims": 30 }, "direzione": "Esplorare il confine: 8 domini GUE, 5 Poisson — il confine è il terzo incluso operativo", "verifica": { "pass": 11, "fail": 0, "skip": 0, "total": 11 }, "fonti_consumate": 0, "fonti_esterne": [ { "id": "video_lp0RgZ6kQF8", "title": "Equivalence between geometrical structures and entropy", "type": "video_digest", "keywords": [ "geometry", "entropy", "symplectic form", "statistical mechanics", "quantum", "thermodynamics", "inner product", "Born rule", "metric tensor", "electromagnetic tensor" ], "content": "La geometria degli stati (classici e quantistici) e l'entropia sono la stessa struttura — invertibili. La forma simplettica conta le configurazioni. Il tensore metrico dello spaziotempo appare dentro la forma simplettica estesa. Il tensore elettromagnetico pure. Statistical mechanics non è costruita sopra alla meccanica — è la stessa cosa.", "teorie": [ "T", "Q", "G", "E" ], "ponti_potenziali": [ { "coppia": "TxQ", "ponte": "forma simplettica = entropia (invertibili)", "nota": "geometry is entropy and entropy is geometry" }, { "coppia": "TxG", "ponte": "tensore metrico dentro la forma simplettica estesa", "nota": "geometria spaziotempo = geometria degli stati in posizione×velocità" }, { "coppia": "ExT", "ponte": "tensore EM dentro la forma simplettica", "nota": "il campo EM conta stati in configurazione posizione×tempo" } ], "timestamp": "2026-04-02T08:23:13.991997" }, { "id": "video_sDlZ-aY9GN4", "title": "Moving charges produce magnetic fields - Einstein relativity", "type": "video_digest", "keywords": [ "magnetic field", "electric field", "length contraction", "time dilation", "Coulomb", "Lorentz", "reference frame", "electromagnetic" ], "content": "Il campo magnetico non esiste come entità separata — è il campo elettrico visto da un altro frame. La contrazione di Lorentz trasforma neutralità in carica. Due elettroni in movimento si separano più lentamente del previsto non per forza magnetica ma per dilatazione temporale. E e B sono manifestazioni dello stesso campo elettromagnetico. La relatività unifica.", "teorie": [ "E", "R" ], "ponti_potenziali": [ { "coppia": "ExR", "ponte": "cambio di frame — E e B sono lo stesso campo", "nota": "il 'ponte' non è l'onda EM, è il cambio di osservatore" } ], "timestamp": "2026-04-02T08:23:13.992016" }, { "id": "video_OwDWOtFNsKQ", "title": "Thermodynamic Computing: Better than Quantum? | Guillaume Verdon (Extropic)", "type": "video_digest", "keywords": [ "thermodynamic computing", "noise", "entropy", "extropic", "fluctuations", "information theory", "probability", "Boltzmann" ], "content": "Computing termodinamico: sfruttare le fluttuazioni termiche invece di combatterle. Il rumore non è nemico — è risorsa. Extropic costruisce hardware che usa l'entropia come motore computazionale. Connessione profonda tra termodinamica, teoria dell'informazione e probabilità.", "teorie": [ "T", "Q" ], "ponti_potenziali": [ { "coppia": "TxQ", "ponte": "noise come risorsa computazionale — fluttuazioni termiche = calcolo", "nota": "il vuoto quantistico (pieno di fluttuazioni) è il computer" } ], "timestamp": "2026-04-02T08:23:13.992019" }, { "id": "video_j0wJBEZdwLs", "title": "What is a Laplace Transform - visual explanation", "type": "video_digest", "keywords": [ "Laplace transform", "frequency", "damping", "s-plane", "complex", "exponential" ], succeeded in 126ms: META: score=0.898 — META: incerto (i=0.5) — massimo potere discriminante G_POTENZIALE_NULLA: score=0.703 — G_POTENZIALE_NULLA: legge di scala da raffinare METRIC_TENSOR: score=0.624 — METRIC_TENSOR: legge di scala da raffinare exec /bin/bash -lc "sed -n '1,260p' tools/data/reports/agent_20260506_0330.md" in /opt/MM_D-ND succeeded in 0ms: # Agent Report — Scale-Selective Perturbations Reveal a Second Axis in GUE, Not in Primes **Date**: 2026-05-06 03:30 **Piano**: 65 **Tension explored**: META (0.5) + BOUNDARY (0.8) ## Claim Under Test > The single latent coordinate found under uniform partial shuffle (PC1 ~99%, observable rank audit 05-05) is an artifact of uniform shuffle collapsing multiple perturbation axes into one. Scale-selective perturbations should reveal multi-dimensional structure. ## Question Do structurally different perturbations (adjacent-swap, block-shuffle, large-gap-only, uniform) produce different observable profiles, or do they all collapse to the same retention curve? ## Experiment Design - **Four perturbation types**: 1. Adjacent-swap: swap neighboring pairs with probability alpha. Destroys lag-1 correlations selectively, preserves long-range order. 2. Block-shuffle: shuffle within blocks of 50 elements. Alpha controls fraction of blocks shuffled. Destroys intra-block order, preserves inter-block structure. 3. Large-gap-only: shuffle positions of above-median gaps only. Preserves small-gap ordering entirely. 4. Uniform partial: baseline from previous runs. - **Alphas**: 0.1, 0.3, 0.5, 0.7, 0.9; 16 trials per (perturbation, alpha) pair. - **Observables**: SR, L1, L2, SR2, triple_var — same set as observable rank audit. - **Null baseline**: 48 full shuffles per domain. - **Domains**: 30,000 prime gaps; GUE (253 unfolded spacings from 23x23 Hermitian matrix). - **PCA**: across all 20 profiles (4 perturbation types x 5 alphas), retention-normalized. Effective rank = exp(entropy of singular value distribution). - **Centroid cosine similarity**: mean retention vector per perturbation type, pairwise cosine. - **Seed**: 20260506. ## Results ### Primes (N=30,000) | Perturbation | SR | L1 | L2 | SR2 | triple_var | |---|---:|---:|---:|---:|---:| | adjacent_swap | 1.039 | 0.886 | 1.399 | 1.003 | 1.246 | | block_shuffle | 0.581 | 0.404 | 0.298 | 0.514 | 0.342 | | large_gap_only | 0.890 | 0.883 | 0.892 | 0.826 | 0.988 | | uniform | 0.307 | 0.376 | 0.376 | 0.358 | 0.237 | *Retention at alpha=0.5. Values >1 mean the perturbation enhanced the signal relative to unperturbed.* PCA: PC1 = 95.4%, PC2 = 2.6%, PC3 = 1.3%. Effective rank = 1.263. All centroid cosine similarities > 0.955. ### GUE (N=253) | Perturbation | SR | L1 | L2 | SR2 | triple_var | |---|---:|---:|---:|---:|---:| | adjacent_swap | 0.918 | 0.784 | -1.135 | 0.941 | 0.985 | | block_shuffle | 0.804 | 0.693 | 0.455 | 0.778 | 0.832 | | large_gap_only | 0.671 | 0.859 | 1.173 | 0.712 | 0.800 | | uniform | 0.449 | 0.423 | 0.634 | 0.353 | 0.435 | *Retention at alpha=0.5.* PCA: PC1 = 73.5%, PC2 = 25.2%, PC3 = 1.1%. Effective rank = 1.889. Adjacent_swap vs large_gap_only cosine = 0.667 (substantially non-aligned). ## Key Findings 1. **GUE has a second perturbation axis that primes do not.** Under scale-selective perturbations, GUE effective rank rises from ~1.02 (uniform-only, rank audit 05-05) to 1.889. The second principal component explains 25.2% of variance. For primes, the rank rises only from 1.07 to 1.26 — the second axis is weak (2.6%). 2. **The L2 observable discriminates perturbation types in GUE.** Under adjacent-swap, GUE L2 retention = -1.135 (sign flip: the perturbation reverses lag-2 correlation). Under large-gap-only, L2 retention = +1.173 (enhancement). This sign difference means adjacent-swap and large-gap-only probe structurally distinct axes of GUE correlations. For primes, L2 also shows anomalous behavior (1.399 under adjacent-swap) but does not flip sign. 3. **Adjacent-swap is the most selective perturbation.** In primes, adjacent swapping barely touches SR (retention 1.039) and SR2 (1.003) while reducing L1 to 0.886 and enhancing L2 to 1.399. The enhancement of L2 under adjacent-swap is not trivial: swapping neighbors creates new lag-2 correlations from the original lag-1 structure (if g_n,g_{n+1} swap, the new g_{n+1} becomes the old g_n, creating a new lag-2 pair from the old lag-1 pair). 4. **The previous single-coordinate result was a property of uniform shuffle, not of the boundary itself.** Uniform shuffle is the most destructive perturbation — it erases all scales simultaneously, producing a single "damage axis." Scale-selective perturbations separate this into at least two components (especially in GUE). 5. **Caveat: GUE sample is small (N=253).** The GUE matrix size was limited by available computation. The effective rank 1.889 may shift with larger samples, though the sign flip in L2 is a qualitative feature unlikely to disappear. ## Verdict **CONSTRAINT on META + BOUNDARY**: the single latent coordinate found under uniform shuffle (rank audit 05-05) is a property of the perturbation type, not of the boundary itself. Scale-selective perturbations reveal a second axis in GUE (PC2=25.2%) and a weak second axis in primes (PC2=2.6%). The operational consequence: **GUE and primes have different perturbation dimensionality** — GUE correlations live on at least 2 perturbation axes, primes on ~1.3. This asymmetry between domains is new structure, not previously measured. Perimeter: 30,000 prime gaps (p_2 to p_{30001}), 253 GUE spacings, 4 perturbation types, 5 alpha values, 16 trials each, seed 20260506. ## Bicono della scoperta - **Due radici**: uniform shuffle (one axis, all scales destroyed) · scale-selective perturbation (multiple axes, each scale targeted separately). The duality is: homogeneous destruction vs structured probing. - **Singolare**: the correlation structure of the sequence before any perturbation is applied. It contains all axes simultaneously — the perturbation type selects which to reveal. - **Invariante di passaggio**: GUE has higher perturbation dimensionality than primes. This holds across the perturbation types tested. The asymmetry (GUE ~2D, primes ~1.3D) persists through the vertex. - **Campo di possibilita**: here it becomes possible to distinguish domains by HOW they respond to structured probing (not just WHETHER they respond). Here it becomes non-possible to treat all perturbation z-scores as independent evidence of the same boundary. ## Auto-audit: 5 lenti - **L1 hard constraint vs bias**: No absolutes. "~2D" and "~1.3D" are effective ranks, not exact integers. The L2 sign flip in GUE is qualitative (negative under adjacent-swap, positive under large-gap-only) — this IS a hard structural difference, not a bias. - **L2 quantita vs ratio**: Retentions are ratios normalized by (original - shuffle_mean). The denominators differ across observables but are used consistently within each observable. No cross-space ratio comparison. - **L3 no silent patching**: The rank audit claim (05-05: "boundary is 1D in observable space") is NOT declared resolved. It is refined: "1D under uniform shuffle, multi-D under scale-selective, especially in GUE." The distinction is explicit. - **L4 edge cases**: GUE N=253 is declared as caveat. The sign flip in L2 is not an edge case — it occurs at all tested alphas (0.1 through 0.9 for adjacent-swap). - **L5 re-discovery**: Scale-selective perturbation is related to multiscale analysis (wavelet-like decompositions). The specific application to prime gap vs GUE spacing perturbation dimensionality is not a standard result in random matrix theory. The L2 sign flip under adjacent-swap may relate to Markov chain perturbation theory. Not tagged NEW — tagged as CONSTRAINT refining previous result. ## Files - Script: `tools/exp_scale_selective_perturbation.py` - Data: `tools/data/scale_selective_perturbation.json` - Report: `tools/data/reports/agent_20260506_0330.md` exec /bin/bash -lc "sed -n '1,220p' tools/data/reports/agent_20260505_1022.md" in /opt/MM_D-ND succeeded in 0ms: # Agent Report — Selective Perturbations Break the Single-Boundary Framing ## Campo Proiezione obbligatoria: ```text META: score=0.898 — META: incerto (i=0.5) — massimo potere discriminante G_POTENZIALE_NULLA: score=0.703 METRIC_TENSOR: score=0.624 ``` La consecutio dagli ultimi tre run era stretta: molti osservabili collassano su una coordinata latente sotto partial-shuffle, e i due layer Markov sembrano rompersi allo stesso confine. La domanda non e' se quel risultato sia falso, ma se il confine misurato sia proprieta' del sistema o proprieta' dell'operatore di perturbazione. ## Esperimento Strumento usato: `tools/exp_selective_layer_decoupling.py` Perimetro atomico: - dati: `primes` con 8000 gap, `GUE` con 175 spacing effettivi prodotti dal generatore corrente, `Poisson` con 8000 spacing; - perturbazioni: `uniform`, `Mk1`, `Mk2`; - alpha: `[0.0, 0.05, ..., 1.0]`; - surrogati per alpha: 8; - soglia operativa: primo alpha dove `|z| >= 2`; - seed: `20260505`. Nota di calcolo: il tentativo `N=60000, n_surr=20` non ha prodotto output in tempo utile; il report usa solo il run completato e dichiara il perimetro ridotto. ## Risultato Nei primi, la perturbazione uniforme non rompe i layer insieme nel perimetro testato: | Observable | Layer | uniform | Mk1 | Mk2 | |---|---:|---:|---:|---:| | SR | L1 | 0.022 | 0.047 | 0.052 | | L1 | L1 | 0.086 | 0.034 | 0.077 | | SR2 | L2 | 0.584 | 0.466 | NEVER | | L2 | L2 | 0.335 | 0.296 | NEVER | | cond_entropy | L2 | 0.006 | 0.008 | 0.006 | | triple_var | L2 | 0.193 | 0.083 | 0.087 | Il dato centrale e' la scissione tra gli osservabili L2: `SR2` e `L2` sono preservati da Mk2 fino ad alpha=1.0, mentre `cond_entropy` e `triple_var` si rompono presto. Quindi "Layer 2" non e' atomico con questa famiglia di osservabili: contiene almeno due sottocanali, uno realmente recuperato dal surrogato Mk2 e uno sensibile alla discretizzazione/varianza della costruzione. Su GUE il campione effettivo e' piccolo (`N=175`), quindi il risultato e' solo indicativo: | Observable | Layer | uniform | Mk1 | Mk2 | |---|---:|---:|---:|---:| | SR | L1 | NEVER | NEVER | NEVER | | L1 | L1 | 0.399 | NEVER | NEVER | | SR2 | L2 | 0.947 | NEVER | NEVER | | L2 | L2 | 0.337 | 0.260 | 0.313 | | cond_entropy | L2 | NEVER | 0.188 | 0.662 | | triple_var | L2 | NEVER | NEVER | 0.350 | Poisson resta quasi nullo rispetto a SR/SR2/triple_var: molte curve non attraversano `|z| >= 2`. Questo e' coerente con un controllo a bassa struttura, non con una prova di assenza assoluta. ## Taglio Il claim precedente "i due Markov layer sono coupled al boundary" va ristretto: **Nel perimetro partial-shuffle dei run precedenti, SR/L1/L2/SR2 mostravano una fase quasi comune. Nel perimetro selective-perturbation di questo run, la fase comune non e' stabile: i critical alpha dei primi separano L1 e L2, e Mk2 preserva SR2/L2 ma non cond_entropy/triple_var.** Questo non falsifica l'esistenza del boundary. Falsifica il framing piu' forte: "un boundary unico per i layer". Il confine osservato e' una superficie operatore-dipendente. Il terzo incluso operativo non e' il valore alpha in se': e' la trasformazione che decide quali correlazioni vengono conservate. ## Vincolo emerso **CONSTRAINT on META + BOUNDARY**: non aggregare gli osservabili in "Layer 1" e "Layer 2" senza dichiarare l'operatore di perturbazione. Nel perimetro `N=8000, n_surr=8`, `cond_entropy` e `triple_var` non sono intercambiabili con `SR2/L2` come prove di Layer 2. Consecutio: il prossimo test deve separare "order-k Markov recovery" da "observable family recovery". La domanda corretta non e' piu' "quanti layer?", ma: quali osservabili sono invarianti rispetto all'operatore Mk2 e quali misurano artefatti di binning/varianza? ## Auto-falsificazione con le 5 lenti L1 hard constraint vs bias: non uso "zero", "impossibile", "mai" come claim assoluti. `NEVER` significa solo "non attraversa `|z| >= 2` sugli alpha testati". L2 quantita' assoluta vs ratio: il confronto usa alpha critici e z-score, non percentuali tra spazi di taglia diversa. GUE ha perimetro ridotto (`N=175`) e non viene pesato come primes/Poisson. L3 no silent patching: il claim precedente viene ristretto esplicitamente. "Coupled boundary" resta valido solo per il partial-shuffle gia' misurato, non come proprieta' generale dei layer. L4 edge case isolation: `cond_entropy` e `triple_var` sono trattati come eccezioni operative dentro L2, non arrotondati dentro la media del layer. L5 re-discovery: il risultato e' metodologico sul lab, non marcato NEW. La nozione vicina e' nota: surrogati Markov di ordine diverso preservano statistiche di ordine diverso; qui il contenuto e' il vincolo sul framing D-ND, non la teoria dei surrogati. ## Output Dati salvati in: ```text tools/data/selective_layer_decoupling.json ``` exec /bin/bash -lc "rg --files tools | rg 'exp_.*\\.py"'$'"' | sort" in /opt/MM_D-ND succeeded in 0ms: tools/exp_3d_boundary_layers.py tools/exp_acf_amplitude_scaling.py tools/exp_acf_range_universality.py tools/exp_acf_stationarity.py tools/exp_acf_z6z_mechanism.py tools/exp_alpha_stability.py tools/exp_beta_crossover.py tools/exp_boundary_coherence.py tools/exp_boundary_growth.py tools/exp_boundary_gue_poisson.py tools/exp_boundary_shuffle_audit.py tools/exp_brody_calibration.py tools/exp_brody_crossover.py tools/exp_brody_flow.py tools/exp_coherence_length.py tools/exp_coherence_robustness.py tools/exp_cross_domain_dipolar_direction.py tools/exp_cross_observable_consistency.py tools/exp_crossover_phase_test.py tools/exp_crossover_universality.py tools/exp_dR_brody_connection.py tools/exp_desitter_unification.py tools/exp_det_drift.py tools/exp_dipolar_angle_reference.py tools/exp_dipolar_crossover.py tools/exp_dipolar_vector_scaling.py tools/exp_excess_scaling.py tools/exp_geodesic_deviation_primes.py tools/exp_magnitude_psd_from_acf.py tools/exp_markov3_observable_hunt.py tools/exp_markov_dipolar_decomposition.py tools/exp_markov_k_direction.py tools/exp_markov_layer_recovery_audit.py tools/exp_markov_memory_by_gue_type.py tools/exp_markov_psd_prediction.py tools/exp_markov_scale_function.py tools/exp_meta_tautology_test.py tools/exp_metric_tensor_diagnostic.py tools/exp_mobius_irrationality.py tools/exp_mod3_scaling.py tools/exp_mod3_vs_residual_ordering.py tools/exp_modular_algebra_depth.py tools/exp_modular_memory_spectrum.py tools/exp_number_variance.py tools/exp_observable_rank_audit.py tools/exp_poisson_convergence.py tools/exp_psd_amplitude_scaling.py tools/exp_psd_prime_gaps.py tools/exp_ricci_desitter_0406.py tools/exp_ricci_primes.py tools/exp_scale_selective_perturbation.py tools/exp_selective_layer_decoupling.py tools/exp_spectral_2d.py tools/exp_spectral_landscape.py tools/exp_spectral_rigidity.py tools/exp_two_channel_boundary.py tools/exp_two_channel_cross_domain.py tools/exp_two_channel_decomposition.py tools/exp_two_channel_psd.py tools/exp_two_channel_shuffle_audit.py tools/exp_two_channel_universality.py tools/exp_two_layer_universality.py codex La dissonanza utile è il caveat del run 03:30: il secondo asse GUE potrebbe essere un effetto del campione piccolo. Taglio quindi su un audit di robustezza per ensemble indipendenti, non su un’altra variante dello stesso partial-shuffle. exec /bin/bash -lc "sed -n '1,260p' tools/exp_scale_selective_perturbation.py" in /opt/MM_D-ND succeeded in 0ms: #!/usr/bin/env python3 """ exp_scale_selective_perturbation.py — Scale-Selective Perturbation Audit Consecutio from observable_rank_audit: under uniform partial shuffle, all observables (SR, L1, L2, SR2, triple_var) collapse to 1 latent coordinate (PC1 ~99%). Is this because the boundary is genuinely 1D, or because uniform shuffle destroys all correlations along the same axis? This experiment applies FOUR structurally different perturbations: 1. Adjacent-swap: swap only neighboring pairs (destroys lag-1, preserves long-range) 2. Block-shuffle: shuffle within blocks of size B (preserves inter-block, destroys intra-block) 3. Large-gap-only: shuffle only the positions of above-median gaps (preserves small-gap ordering) 4. Uniform partial: the baseline from previous runs (for comparison) For each perturbation at multiple intensities, measure SR/L1/L2/SR2/triple_var, compute PCA across perturbation types, and check if the rank is > 1. If different perturbation types produce different observable profiles -> the boundary is multi-dimensional and uniform shuffle was collapsing it. If all perturbation types produce the same profile -> the boundary is 1D in observable space. Usage: python tools/exp_scale_selective_perturbation.py [--N 30000] [--seed 20260506] """ import argparse import json import numpy as np from pathlib import Path from sympy import nextprime def generate_primes(N): """Generate first N prime gaps.""" gaps = [] p = 2 for _ in range(N + 1): p_next = nextprime(p) gaps.append(p_next - p) p = p_next if len(gaps) >= N: break return np.array(gaps, dtype=float) def generate_gue(N, rng): """Generate N GUE gaps (eigenvalue spacings of random Hermitian matrix).""" dim = int(np.sqrt(2 * N)) + 10 H = rng.standard_normal((dim, dim)) + 1j * rng.standard_normal((dim, dim)) H = (H + H.conj().T) / 2 evals = np.sort(np.linalg.eigvalsh(H)) spacings = np.diff(evals) spacings = spacings[spacings > 0] # Unfold: normalize by local mean from scipy.ndimage import uniform_filter1d local_mean = uniform_filter1d(spacings.astype(float), size=max(50, len(spacings)//20)) local_mean[local_mean < 1e-12] = 1e-12 unfolded = spacings / local_mean if len(unfolded) >= N: return unfolded[:N] return unfolded def generate_poisson(N, rng): """Generate N Poisson (iid exponential) gaps.""" return rng.exponential(1.0, size=N) # ---------- Observables ---------- def spectral_rigidity(gaps, L=10): """Delta_3(L) spectral rigidity.""" cumulative = np.cumsum(gaps) cumulative = cumulative / cumulative[-1] * len(cumulative) n = np.arange(1, len(cumulative) + 1, dtype=float) window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) if window < 5: return 0.0 residuals = [] for start in range(0, len(gaps) - window, max(1, window // 2)): seg_n = n[start:start+window] seg_c = cumulative[start:start+window] coeffs = np.polyfit(seg_n, seg_c, 1) fitted = np.polyval(coeffs, seg_n) residuals.append(np.mean((seg_c - fitted)**2)) return float(np.mean(residuals)) def lag1_autocorr(gaps): """Lag-1 autocorrelation.""" g = gaps - np.mean(gaps) c0 = np.dot(g, g) if c0 < 1e-15: return 0.0 return float(np.dot(g[:-1], g[1:]) / c0) def lag2_autocorr(gaps): """Lag-2 autocorrelation.""" g = gaps - np.mean(gaps) c0 = np.dot(g, g) if c0 < 1e-15: return 0.0 return float(np.dot(g[:-2], g[2:]) / c0) def sr2(gaps, L=20): """Spectral rigidity at larger L.""" return spectral_rigidity(gaps, L=L) def triple_variance(gaps): """Variance of triple products g_n * g_{n+1} * g_{n+2}.""" triples = gaps[:-2] * gaps[1:-1] * gaps[2:] return float(np.var(triples)) def compute_observables(gaps): """Compute all 5 observables.""" return { 'SR': spectral_rigidity(gaps), 'L1': lag1_autocorr(gaps), 'L2': lag2_autocorr(gaps), 'SR2': sr2(gaps), 'triple_var': triple_variance(gaps), } # ---------- Perturbation types ---------- def perturb_adjacent_swap(gaps, alpha, rng): """Swap adjacent pairs with probability alpha. Destroys lag-1, preserves long-range.""" g = gaps.copy() n = len(g) for i in range(0, n - 1, 2): if rng.random() < alpha: g[i], g[i+1] = g[i+1], g[i] return g def perturb_block_shuffle(gaps, alpha, rng, block_size=50): """Shuffle within blocks of size B. Alpha controls fraction of blocks shuffled.""" g = gaps.copy() n = len(g) n_blocks = n // block_size blocks_to_shuffle = rng.choice(n_blocks, size=max(1, int(alpha * n_blocks)), replace=False) for b in blocks_to_shuffle: start = b * block_size end = min(start + block_size, n) rng.shuffle(g[start:end]) return g def perturb_large_gap_only(gaps, alpha, rng): """Shuffle only above-median gap positions. Alpha controls fraction shuffled.""" g = gaps.copy() median_val = np.median(g) large_idx = np.where(g > median_val)[0] n_shuffle = max(2, int(alpha * len(large_idx))) chosen = rng.choice(large_idx, size=min(n_shuffle, len(large_idx)), replace=False) vals = g[chosen].copy() rng.shuffle(vals) g[chosen] = vals return g def perturb_uniform(gaps, alpha, rng): """Standard uniform partial shuffle. Baseline from previous runs.""" g = gaps.copy() n = len(g) n_swap = max(1, int(alpha * n / 2)) for _ in range(n_swap): i, j = rng.integers(0, n, size=2) g[i], g[j] = g[j], g[i] return g PERTURBATION_TYPES = { 'adjacent_swap': perturb_adjacent_swap, 'block_shuffle': perturb_block_shuffle, 'large_gap_only': perturb_large_gap_only, 'uniform': perturb_uniform, } # ---------- Main experiment ---------- def run_experiment(N=30000, seed=20260506, n_trials=16): rng = np.random.default_rng(seed) alphas = [0.1, 0.3, 0.5, 0.7, 0.9] obs_names = ['SR', 'L1', 'L2', 'SR2', 'triple_var'] results = {} for domain_name, gen_func in [('primes', lambda: generate_primes(N)), ('GUE', lambda: generate_gue(N, rng))]: gaps = gen_func() actual_n = len(gaps) print(f"\n=== {domain_name} (N={actual_n}) ===") original_obs = compute_observables(gaps) print(f"Original: {original_obs}") # Full shuffle baseline (z-score reference) full_shuffle_obs = {k: [] for k in obs_names} for _ in range(48): g_shuf = gaps.copy() rng.shuffle(g_shuf) obs = compute_observables(g_shuf) for k in obs_names: full_shuffle_obs[k].append(obs[k]) full_mean = {k: np.mean(v) for k, v in full_shuffle_obs.items()} full_std = {k: np.std(v) for k, v in full_shuffle_obs.items()} # For each perturbation type, collect observable retention curves # retention = (obs_perturbed - obs_shuffle) / (obs_original - obs_shuffle) domain_result = { 'original': original_obs, 'full_shuffle_mean': full_mean, 'full_shuffle_std': full_std, 'perturbations': {}, } # Collect: for each (perturbation_type, alpha) -> mean observable vector # Then do PCA across perturbation types all_profiles = [] # list of (pert_type, alpha, obs_vector) for pert_name, pert_func in PERTURBATION_TYPES.items(): pert_result = {} for alpha in alphas: trial_obs = {k: [] for k in obs_names} for t in range(n_trials): g_pert = pert_func(gaps, alpha, rng) obs = compute_observables(g_pert) for k in obs_names: trial_obs[k].append(obs[k]) mean_obs = {k: float(np.mean(v)) for k, v in trial_obs.items()} # Compute retention: how much of the original signal survives retention = {} for k in obs_names: denom = original_obs[k] - full_mean[k] if abs(denom) < 1e-15: retention[k] = 0.0 else: retention[k] = (mean_obs[k] - full_mean[k]) / denom pert_result[f'alpha_{alpha}'] = { 'mean': mean_obs, 'retention': retention, } obs_vec = [retention[k] for k in obs_names] all_profiles.append((pert_name, alpha, obs_vec)) domain_result['perturbations'][pert_name] = pert_result # PCA across all (perturbation_type, alpha) profiles exec /bin/bash -lc "sed -n '1,220p' tools/exp_observable_rank_audit.py" in /opt/MM_D-ND succeeded in 0ms: #!/usr/bin/env python3 """ exp_observable_rank_audit.py - META audit for observable redundancy. Recent runs found many Markov/crossover observables that react coherently under partial shuffle. This script asks whether those observables carry independent directions or mostly re-measure one latent boundary coordinate. It measures retention curves from alpha-partial shuffles, then reports: - original-vs-full-shuffle z for each observable - PCA energy of the retention matrix across alpha - effective rank of that matrix - pairwise correlations between retention curves The script measures data only. The report decides the structural claim. """ import argparse import json from pathlib import Path import numpy as np from exp_3d_boundary_layers import get_primes, gue_gaps, partial_shuffle def obs_spacing_ratio(gaps): r = np.minimum(gaps[:-1], gaps[1:]) / np.maximum(gaps[:-1], gaps[1:]) r = r[np.isfinite(r)] return float(np.mean(r)) if len(r) else 0.0 def obs_lag_acf(gaps, lag): g = gaps - np.mean(gaps) c0 = np.mean(g * g) if c0 == 0: return 0.0 return float(np.mean(g[:-lag] * g[lag:]) / c0) def obs_sr2(gaps): r = np.minimum(gaps[:-2], gaps[2:]) / np.maximum(gaps[:-2], gaps[2:]) r = r[np.isfinite(r)] return float(np.mean(r)) if len(r) else 0.0 def obs_triple_var(gaps): triples = gaps[:-2] + gaps[1:-1] + gaps[2:] v = np.var(gaps) if v == 0: return 0.0 return float(np.var(triples) / v) OBSERVABLES = { "SR": obs_spacing_ratio, "L1": lambda gaps: obs_lag_acf(gaps, 1), "L2": lambda gaps: obs_lag_acf(gaps, 2), "SR2": obs_sr2, "triple_var": obs_triple_var, } def measure(gaps): return {name: fn(gaps) for name, fn in OBSERVABLES.items()} def full_shuffle_baseline(gaps, n_trials, rng): vals = {name: [] for name in OBSERVABLES} for _ in range(n_trials): s = rng.permutation(gaps) row = measure(s) for name, value in row.items(): vals[name].append(value) return { name: { "mean": float(np.mean(x)), "std": float(np.std(x, ddof=1)) if len(x) > 1 else 0.0, } for name, x in vals.items() } def retention_curves(gaps, alphas, n_trials, originals, baseline, rng): rows = [] for alpha in alphas: vals = {name: [] for name in OBSERVABLES} for _ in range(n_trials): s = partial_shuffle(gaps, float(alpha), rng) row = measure(s) for name, value in row.items(): vals[name].append(value) out = {"alpha": float(alpha)} for name in OBSERVABLES: mean = float(np.mean(vals[name])) denom = originals[name] - baseline[name]["mean"] retention = (mean - baseline[name]["mean"]) / denom if abs(denom) > 1e-12 else 0.0 out[name] = { "mean": mean, "std": float(np.std(vals[name], ddof=1)) if len(vals[name]) > 1 else 0.0, "retention": float(retention), } rows.append(out) return rows def pca_summary(rows): names = list(OBSERVABLES) matrix = np.array([[row[name]["retention"] for name in names] for row in rows], dtype=float) matrix = matrix - np.mean(matrix, axis=0, keepdims=True) _, singular, vt = np.linalg.svd(matrix, full_matrices=False) energy = singular * singular if np.sum(energy) <= 1e-15: explained = np.zeros_like(energy) effective_rank = 0.0 else: explained = energy / np.sum(energy) positive = explained[explained > 1e-15] effective_rank = float(np.exp(-np.sum(positive * np.log(positive)))) corr = np.corrcoef(matrix, rowvar=False) abs_corr = np.abs(corr) upper = abs_corr[np.triu_indices_from(abs_corr, k=1)] return { "observables": names, "singular_values": [float(x) for x in singular], "explained_variance": [float(x) for x in explained], "effective_rank": effective_rank, "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(names)} if len(vt) else {}, "mean_abs_pairwise_corr": float(np.mean(upper)) if len(upper) else 0.0, "min_abs_pairwise_corr": float(np.min(upper)) if len(upper) else 0.0, "max_abs_pairwise_corr": float(np.max(upper)) if len(upper) else 0.0, } def analyze_sequence(name, gaps, alphas, n_trials, n_baseline, rng): originals = measure(gaps) baseline = full_shuffle_baseline(gaps, n_baseline, rng) rows = retention_curves(gaps, alphas, n_trials, originals, baseline, rng) z = {} for obs_name in OBSERVABLES: std = baseline[obs_name]["std"] z[obs_name] = float((originals[obs_name] - baseline[obs_name]["mean"]) / std) if std > 1e-12 else 0.0 return { "n_gaps": int(len(gaps)), "originals": originals, "full_shuffle_baseline": baseline, "original_vs_shuffle_z": z, "retention_curves": rows, "pca": pca_summary(rows), } def build_sequences(n_gaps, rng): primes = get_primes(n_gaps * 24)[: n_gaps + 1] prime_gaps = np.diff(primes).astype(float) gue = gue_gaps(160, max(8, n_gaps // 160 + 1), rng).astype(float) gue = gue[:n_gaps] poisson = rng.exponential(1.0, size=n_gaps).astype(float) return { "primes": prime_gaps, "gue": gue, "poisson": poisson, } def run(n_gaps=30000, n_alpha=19, n_trials=24, n_baseline=72, seed=20260505): rng = np.random.default_rng(seed) alphas = np.linspace(0.05, 0.95, n_alpha) sequences = build_sequences(n_gaps, rng) output = { "experiment": "observable_rank_audit", "question": "Do crossover observables contain independent directions or one latent boundary coordinate?", "params": { "n_gaps": int(n_gaps), "n_alpha": int(n_alpha), "n_trials": int(n_trials), "n_baseline": int(n_baseline), "seed": int(seed), }, "sequences": {}, } print(f"n_gaps={n_gaps}, n_alpha={n_alpha}, n_trials={n_trials}, n_baseline={n_baseline}, seed={seed}") print(f"{'sequence':<10} {'pc1':>8} {'eff_rank':>9} {'mean|corr|':>11} z(SR,L1,L2,SR2,triple)") print("-" * 92) for seq_name, gaps in sequences.items(): result = analyze_sequence(seq_name, gaps, alphas, n_trials, n_baseline, rng) output["sequences"][seq_name] = result pca = result["pca"] z = result["original_vs_shuffle_z"] z_text = ", ".join(f"{obs}={z[obs]:+.1f}" for obs in OBSERVABLES) pc1 = pca["explained_variance"][0] if pca["explained_variance"] else 0.0 print(f"{seq_name:<10} {pc1:>8.3f} {pca['effective_rank']:>9.3f} {pca['mean_abs_pairwise_corr']:>11.3f} {z_text}") out_path = Path("tools/data/observable_rank_audit.json") with out_path.open("w") as f: json.dump(output, f, indent=2) print(f"\nSaved to {out_path}") return output def main(): parser = argparse.ArgumentParser() parser.add_argument("--n_gaps", type=int, default=30000) parser.add_argument("--n_alpha", type=int, default=19) parser.add_argument("--n_trials", type=int, default=24) parser.add_argument("--n_baseline", type=int, default=72) parser.add_argument("--seed", type=int, default=20260505) args = parser.parse_args() run( n_gaps=args.n_gaps, exec /bin/bash -lc 'rg "def .*gue|gue|hermitian|eigvalsh|unfold" tools/exp_*.py' in /opt/MM_D-ND succeeded in 0ms: tools/exp_boundary_gue_poisson.py: r_gue = 0.5307 # GOE (real symmetric) in 1D tools/exp_boundary_gue_poisson.py: print(f"\nReference: _GUE = {r_gue:.4f}, _Poisson = {r_poisson:.4f}") tools/exp_boundary_gue_poisson.py: dist_gue = abs(rp - r_gue) tools/exp_boundary_gue_poisson.py: label = "GUE" if dist_gue < dist_poi else "POISSON" tools/exp_boundary_gue_poisson.py: margin = abs(dist_gue - dist_poi) tools/exp_boundary_gue_poisson.py: "experiment": "boundary_gue_poisson_cramer", tools/exp_boundary_gue_poisson.py: "reference": {"r_gue": r_gue, "r_poisson": r_poisson}, tools/exp_brody_flow.py: """MLE estimate of Brody parameter beta from unfolded spacings. tools/exp_brody_flow.py:def unfold_spacings(gaps): tools/exp_brody_flow.py: uf = unfold_spacings(g) tools/exp_brody_flow.py: uf_shuf = unfold_spacings(g_shuf) tools/exp_brody_flow.py: uf_cr = unfold_spacings(g_cr) tools/exp_3d_boundary_layers.py:def gue_gaps(N_mat, n_matrices, rng): tools/exp_3d_boundary_layers.py: evals = np.sort(np.linalg.eigvalsh(H)) tools/exp_3d_boundary_layers.py: gue_g = gue_gaps(n_mat, n_matrices, rng) tools/exp_3d_boundary_layers.py: if len(gue_g) > args.N: tools/exp_3d_boundary_layers.py: gue_g = gue_g[:args.N] tools/exp_3d_boundary_layers.py: gue_results, gue_orig, gue_bl_mean, gue_bl_std = run_crossover( tools/exp_3d_boundary_layers.py: gue_g, alphas, args.n_trials, rng, "GUE" tools/exp_3d_boundary_layers.py: ('gue', gue_results, gue_orig, gue_bl_mean, gue_bl_std), tools/exp_3d_boundary_layers.py: gue_sep = output['sequences']['gue']['layer_separation']['delta'] tools/exp_3d_boundary_layers.py: print(f"Layer separation Δα: Primes={prime_sep:+.3f}, GUE={gue_sep:+.3f}, Poisson={pois_sep:+.3f}") tools/exp_3d_boundary_layers.py: 'gue_layer_separation': float(gue_sep), tools/exp_cross_observable_consistency.py:def unfold_primes(p): tools/exp_cross_observable_consistency.py:def number_variance_at_L(unfolded, L, n_samples=3000): tools/exp_cross_observable_consistency.py: x_min, x_max = unfolded[0], unfolded[-1] tools/exp_cross_observable_consistency.py: counts = np.array([np.sum((unfolded >= x0) & (unfolded < x0 + L)) for x0 in starts]) tools/exp_cross_observable_consistency.py:def gue_gaps(n_eigenvalues=2000, n_matrices=5): tools/exp_cross_observable_consistency.py: evals = np.sort(np.linalg.eigvalsh(H).real) tools/exp_cross_observable_consistency.py:unfolded = unfold_primes(primes) tools/exp_cross_observable_consistency.py: s2 = number_variance_at_L(unfolded, L) tools/exp_cross_observable_consistency.py: u_shuf = unfold_primes(p_shuf) tools/exp_cross_observable_consistency.py:gue_g = gue_gaps(n_eigenvalues=1500, n_matrices=4) tools/exp_cross_observable_consistency.py:r_gue = r_statistic(gue_g) tools/exp_cross_observable_consistency.py:beta_r_gue = beta_from_r(r_gue) tools/exp_cross_observable_consistency.py:print(f"r = {r_gue:.6f} → β_r = {beta_r_gue:.3f}") tools/exp_cross_observable_consistency.py:# For GUE, unfold eigenvalues directly tools/exp_cross_observable_consistency.py: evals = np.sort(np.linalg.eigvalsh(H).real) tools/exp_cross_observable_consistency.py:beta_sig_gue = {} tools/exp_cross_observable_consistency.py: beta_sig_gue[L] = b tools/exp_cross_observable_consistency.py:vals_gue = [f"{beta_sig_gue[L]:.3f}" for L in L_values] tools/exp_cross_observable_consistency.py:print(f"{'GUE':<12} {beta_r_gue:>6.3f} | " + " | ".join(f"{v:>9}" for v in vals_gue)) tools/exp_cross_observable_consistency.py:disagree_gue = max(abs(beta_r_gue - beta_sig_gue[L]) for L in L_values) tools/exp_cross_observable_consistency.py:print(f" GUE: {disagree_gue:.3f}") tools/exp_cross_observable_consistency.py: "gue": { tools/exp_cross_observable_consistency.py: "r": float(r_gue), tools/exp_cross_observable_consistency.py: "beta_r": float(beta_r_gue), tools/exp_cross_observable_consistency.py: "beta_sigma": {str(L): float(beta_sig_gue[L]) for L in L_values}, tools/exp_cross_observable_consistency.py: "max_disagreement": float(disagree_gue), tools/exp_boundary_coherence.py: "spacing_ratio": {"poisson": 0.38629, "gue": 0.53590}, # 2ln2-1, 4-2√3 tools/exp_boundary_coherence.py: "gap_var_ratio": {"poisson": 1.0, "gue": 0.178}, tools/exp_boundary_coherence.py: "small_gap_frac": {"poisson": 0.2592, "gue": 0.020}, # P(s<0.3) for exp vs Wigner tools/exp_boundary_coherence.py: "brody_beta": {"poisson": 0.0, "gue": 1.0}, tools/exp_boundary_coherence.py: "lag1_acf": {"poisson": 0.0, "gue": -0.271}, tools/exp_boundary_coherence.py: """Normalize gaps to mean 1 (unfolding).""" tools/exp_boundary_coherence.py: g = REF[obs_name]["gue"] tools/exp_boundary_coherence.py:def generate_gue_spacings(n, n_matrices=50): tools/exp_boundary_coherence.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_boundary_coherence.py: gue_gaps = generate_gue_spacings(20000) tools/exp_boundary_coherence.py: gue_obs = compute_all_observables(gue_gaps) tools/exp_boundary_coherence.py: "raw": gue_obs, tools/exp_boundary_coherence.py: "tau": {k: to_tau(k, v) for k, v in gue_obs.items()}, tools/exp_cross_domain_dipolar_direction.py: eigs = np.linalg.eigvalsh(H) tools/exp_cross_domain_dipolar_direction.py: # Unfold: divide by local mean spacing (simple unfolding) tools/exp_cross_domain_dipolar_direction.py: """Uniform iid on [0,2] — mean 1, same as unfolded.""" tools/exp_acf_range_universality.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_acf_range_universality.py: gue_gaps = generate_rmt_spacings(N_rmt, n_mat_rmt, 'GUE') tools/exp_acf_range_universality.py: 'GUE': gue_gaps, tools/exp_brody_calibration.py: """Sigma^2(L) = Var[N(x, x+L)] for unfolded levels.""" tools/exp_brody_calibration.py:def prime_gaps_unfolded(n_gaps): tools/exp_brody_calibration.py:def gue_gaps(n_gaps, rng): tools/exp_brody_calibration.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_brody_calibration.py: unfolded = gaps_raw * local_density tools/exp_brody_calibration.py: margin = len(unfolded) // 10 tools/exp_brody_calibration.py: unfolded = unfolded[margin:-margin] tools/exp_brody_calibration.py: return unfolded[:n_gaps] tools/exp_brody_calibration.py: prime_gaps = prime_gaps_unfolded(args.n_gaps) tools/exp_brody_calibration.py: gue_g = gue_gaps(min(args.n_gaps, 400), rng) tools/exp_brody_calibration.py: obs_g = compute_observables(gue_g, n_shuffles=args.n_shuffles, rng=rng) tools/exp_brody_calibration.py: real_domains['gue_matrix'] = {**obs_g, 'beta_eff': beta_eff_g} tools/exp_brody_calibration.py: print(f"{'gue_matrix':>20} {obs_g['r']:8.4f} {obs_g['r_shuf']:8.4f} " tools/exp_markov_memory_by_gue_type.py:def generate_gue_gaps(n=2000): tools/exp_markov_memory_by_gue_type.py: from scipy.linalg import eigvalsh tools/exp_markov_memory_by_gue_type.py: evals = eigvalsh(H) tools/exp_markov_memory_by_gue_type.py: domains['gue_matrix'] = { tools/exp_markov_memory_by_gue_type.py: 'gaps': generate_gue_gaps(3000), tools/exp_markov_memory_by_gue_type.py: 'gue_type': info['type'], tools/exp_markov_memory_by_gue_type.py: for r in sorted(results, key=lambda x: x['gue_type']): tools/exp_markov_memory_by_gue_type.py: print(f"{r['domain']:<22} {r['gue_type']:<18} {r['N']:>6} " tools/exp_markov_memory_by_gue_type.py: subset = [r for r in results if r['gue_type'] == gtype] tools/exp_markov_memory_by_gue_type.py: 'experiment': 'markov_memory_by_gue_type', tools/exp_markov_memory_by_gue_type.py: outpath = '/opt/MM_D-ND/tools/data/markov_memory_by_gue_type.json' tools/exp_dipolar_crossover.py:def gue_spacings(N_mat, n_matrices, rng): tools/exp_dipolar_crossover.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_dipolar_crossover.py: gue_mats = gue_spacings(N_mat, n_matrices, rng) tools/exp_dipolar_crossover.py: sr0, l1_0, _, _ = compute_dipolar(gue_mats) tools/exp_dipolar_crossover.py: for s in gue_mats: tools/exp_dipolar_crossover.py: trial_mats = [partial_shuffle(s, alpha, rng_trial) for s in gue_mats] tools/exp_coherence_robustness.py:Segue agent_20260416_0330 (COHERENCE_LENGTH). Obiettivo: stimare confidence intervals tools/exp_beta_crossover.py: from scipy.linalg import eigvalsh_tridiagonal tools/exp_beta_crossover.py: eigs = eigvalsh_tridiagonal(diag, off_diag) tools/exp_beta_crossover.py: # Trim edges (unfolding artifacts) tools/exp_dipolar_angle_reference.py:1. Pure GUE eigenvalue spacings (GOE/GUE unfolded) tools/exp_dipolar_angle_reference.py:from numpy.linalg import eigvalsh tools/exp_dipolar_angle_reference.py:def generate_gue_gaps(n_gaps, matrix_size=500): tools/exp_dipolar_angle_reference.py: """Generate gaps from GUE eigenvalues (unfolded).""" tools/exp_dipolar_angle_reference.py: eigs = np.sort(eigvalsh(H)) tools/exp_dipolar_angle_reference.py: """Generate gaps from GOE eigenvalues (unfolded).""" tools/exp_dipolar_angle_reference.py: eigs = np.sort(eigvalsh(H)) tools/exp_dipolar_angle_reference.py: gue_thetas = [] tools/exp_dipolar_angle_reference.py: gue_data = [] tools/exp_dipolar_angle_reference.py: gue_gaps = generate_gue_gaps(len(prime_gaps)) tools/exp_dipolar_angle_reference.py: theta, dsr, dl1, sr, l1, srs, l1s = dipolar_angle(gue_gaps, n_shuffle=50) tools/exp_dipolar_angle_reference.py: gue_thetas.append(theta) tools/exp_dipolar_angle_reference.py: gue_data.append((theta, dsr, dl1, sr, l1)) tools/exp_dipolar_angle_reference.py: gue_thetas = np.array(gue_thetas) tools/exp_dipolar_angle_reference.py: 'theta_mean': np.mean(gue_thetas), 'theta_std': np.std(gue_thetas), tools/exp_dipolar_angle_reference.py: 'thetas': gue_thetas.tolist(), tools/exp_dipolar_angle_reference.py: 'SR_mean': np.mean([d[3] for d in gue_data]), tools/exp_dipolar_angle_reference.py: 'L1_mean': np.mean([d[4] for d in gue_data]), tools/exp_dipolar_angle_reference.py: print(f" GUE: theta = {np.mean(gue_thetas):.1f} +/- {np.std(gue_thetas):.1f} deg") tools/exp_mod3_vs_residual_ordering.py:def unfold_gaps(gaps): tools/exp_mod3_vs_residual_ordering.py:def number_variance(unfolded_gaps, L_values, n_starts=3000): tools/exp_mod3_vs_residual_ordering.py: """Sigma^2(L) from unfolded gaps.""" tools/exp_mod3_vs_residual_ordering.py: levels = np.cumsum(unfolded_gaps) tools/exp_mod3_vs_residual_ordering.py: unfolded = unfold_gaps(gaps.astype(float)) tools/exp_mod3_vs_residual_ordering.py: print(f"Unfolded: mean={np.mean(unfolded):.3f}, std={np.std(unfolded):.3f}") tools/exp_mod3_vs_residual_ordering.py: sig2_real = number_variance(unfolded, L_values) tools/exp_mod3_vs_residual_ordering.py: shuf = unfolded.copy() tools/exp_mod3_vs_residual_ordering.py: # Shuffle raw gaps with mod-3 constraint, then unfold tools/exp_mod3_vs_residual_ordering.py: shuf_unf = unfold_gaps(shuf_raw.astype(float)) tools/exp_mod3_vs_residual_ordering.py: cu = unfold_gaps(cg) tools/exp_boundary_shuffle_audit.py:def gen_gue_eigenvalues(size=2000, n_matrices=50): tools/exp_boundary_shuffle_audit.py: eigs = np.sort(linalg.eigvalsh(H)) tools/exp_boundary_shuffle_audit.py: eigs = np.sort(linalg.eigvalsh(H)) tools/exp_boundary_shuffle_audit.py: eigs = np.sort(linalg.eigvalsh(K)) tools/exp_boundary_shuffle_audit.py: 'gue': ('GUE random matrix', gen_gue_eigenvalues), tools/exp_boundary_shuffle_audit.py: dist_gue = abs(res['r_original'] - R_GUE) tools/exp_boundary_shuffle_audit.py: res['class_original'] = 'GUE' if dist_gue < dist_poi else 'Poisson' tools/exp_boundary_shuffle_audit.py: dist_gue_s = abs(res['r_shuffled_mean'] - R_GUE) tools/exp_boundary_shuffle_audit.py: res['class_shuffled'] = 'GUE' if dist_gue_s < dist_poi_s else 'Poisson' tools/exp_observable_rank_audit.py:from exp_3d_boundary_layers import get_primes, gue_gaps, partial_shuffle tools/exp_observable_rank_audit.py: gue = gue_gaps(160, max(8, n_gaps // 160 + 1), rng).astype(float) tools/exp_observable_rank_audit.py: gue = gue[:n_gaps] tools/exp_observable_rank_audit.py: "gue": gue, tools/exp_spectral_landscape.py: """Prime gaps (unfolded via local density).""" tools/exp_spectral_landscape.py:def gen_gue(n_spacings): tools/exp_spectral_landscape.py: eigs = np.sort(np.linalg.eigvalsh(H).real) tools/exp_spectral_landscape.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_spectral_landscape.py: eigs = np.sort(np.linalg.eigvalsh(H).real) tools/exp_spectral_landscape.py: s_chaotic = gen_gue(n_chaotic) if n_chaotic > 100 else np.array([]) tools/exp_spectral_landscape.py: """Gaps between Fibonacci numbers (unfolded).""" tools/exp_spectral_landscape.py: """Gaps between quadratic residues mod large prime (unfolded).""" tools/exp_spectral_landscape.py: return gen_gue(n_spacings) # theoretically identical tools/exp_spectral_landscape.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_spectral_landscape.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_spectral_landscape.py: ("GUE_matrix", gen_gue, {}), tools/exp_crossover_phase_test.py:def generate_gue_gaps(N, rng): tools/exp_crossover_phase_test.py: evals = np.sort(np.linalg.eigvalsh(H)) tools/exp_crossover_phase_test.py: sequences['GUE'] = generate_gue_gaps(args.N, rng) tools/exp_spectral_2d.py: """Prime gaps, unfolded by local density.""" tools/exp_spectral_2d.py: """Raw prime gaps (not unfolded) for distribution sampling.""" tools/exp_spectral_2d.py:def gen_gue(n_spacings): tools/exp_spectral_2d.py: eigs = np.sort(np.linalg.eigvalsh(H).real) tools/exp_spectral_2d.py: s_chaotic = gen_gue(n_chaotic) if n_chaotic > 50 else np.random.exponential(1.0, n_chaotic) tools/exp_spectral_2d.py: ("GUE", lambda n: gen_gue(n)), tools/exp_two_layer_universality.py:def gen_gue_spacings(N, rng=None): tools/exp_two_layer_universality.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_two_layer_universality.py: 'GUE': gen_gue_spacings, tools/exp_number_variance.py:def unfolded_primes(primes): tools/exp_number_variance.py: unfolded = p / np.log(p) tools/exp_number_variance.py: spacings = np.diff(unfolded) tools/exp_number_variance.py: unfolded = unfolded / mean_s tools/exp_number_variance.py: return unfolded tools/exp_number_variance.py:def number_variance(unfolded, L_values): tools/exp_number_variance.py: x_min, x_max = unfolded[0], unfolded[-1] tools/exp_number_variance.py: n = np.sum((unfolded >= x0) & (unfolded < x0 + L)) tools/exp_number_variance.py:def number_variance_gue(L_values): tools/exp_number_variance.py: uf = unfolded_primes(primes) tools/exp_number_variance.py: sv_gue = number_variance_gue(L_VALUES) tools/exp_number_variance.py: coeffs_gue = np.polyfit(log_L, sv_gue, 1) tools/exp_number_variance.py: print(f" {L:4d} | {sv_primes[i]:8.4f} | {sv_shuffled[i]:8.4f} | {sv_gue[i]:7.4f} | {sv_poisson[i]:8.1f}") tools/exp_number_variance.py: "gue_log_slope": float(2/np.pi**2), tools/exp_two_channel_cross_domain.py: python tools/exp_two_channel_cross_domain.py [--n_primes N] [--gue_size N] [--n_windows N] tools/exp_two_channel_cross_domain.py:def gue_eigenvalues(n_matrices, matrix_size): tools/exp_two_channel_cross_domain.py: """Generate unfolded spacings from GUE random matrices.""" tools/exp_two_channel_cross_domain.py: evals = np.sort(np.linalg.eigvalsh(H)) tools/exp_two_channel_cross_domain.py: # Local unfolding tools/exp_two_channel_cross_domain.py:def run(n_primes=200000, gue_matrices=50, gue_size=800, n_windows=8, window=5000, n_surrogates=20): tools/exp_two_channel_cross_domain.py: print(f"\n=== GUE EIGENVALUES ({gue_matrices} matrices of size {gue_size}) ===") tools/exp_two_channel_cross_domain.py: gue_spacings = gue_eigenvalues(gue_matrices, gue_size) tools/exp_two_channel_cross_domain.py: print(f"Got {len(gue_spacings)} GUE spacings, mean={np.mean(gue_spacings):.3f}") tools/exp_two_channel_cross_domain.py: gue_max_start = len(gue_spacings) - window - 10 tools/exp_two_channel_cross_domain.py: gue_starts = np.unique(np.logspace(1, np.log10(max(gue_max_start, 100)), n_windows).astype(int)) tools/exp_two_channel_cross_domain.py: gue_starts = gue_starts[gue_starts < gue_max_start] tools/exp_two_channel_cross_domain.py: gue_results = [] tools/exp_two_channel_cross_domain.py: for s in gue_starts: tools/exp_two_channel_cross_domain.py: gw = gue_spacings[s:s + window] tools/exp_two_channel_cross_domain.py: gue_results.append(obs) tools/exp_two_channel_cross_domain.py: results['gue'] = gue_results tools/exp_two_channel_cross_domain.py: parser.add_argument('--gue_matrices', type=int, default=50) tools/exp_two_channel_cross_domain.py: parser.add_argument('--gue_size', type=int, default=800) tools/exp_two_channel_cross_domain.py: results = run(args.n_primes, args.gue_matrices, args.gue_size, tools/exp_two_channel_cross_domain.py: 'gue_matrices': args.gue_matrices, tools/exp_two_channel_cross_domain.py: 'gue_size': args.gue_size, tools/exp_selective_layer_decoupling.py:def gen_gue_spacings(N, rng): tools/exp_selective_layer_decoupling.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_selective_layer_decoupling.py: 'GUE': gen_gue_spacings(args.N, rng), tools/exp_scale_selective_perturbation.py:def generate_gue(N, rng): tools/exp_scale_selective_perturbation.py: evals = np.sort(np.linalg.eigvalsh(H)) tools/exp_scale_selective_perturbation.py: unfolded = spacings / local_mean tools/exp_scale_selective_perturbation.py: if len(unfolded) >= N: tools/exp_scale_selective_perturbation.py: return unfolded[:N] tools/exp_scale_selective_perturbation.py: return unfolded tools/exp_scale_selective_perturbation.py: ('GUE', lambda: generate_gue(N, rng))]: tools/exp_spectral_rigidity.py:def generate_gue_gaps(n=600): tools/exp_spectral_rigidity.py: eigs = np.sort(np.linalg.eigvalsh(H)) tools/exp_spectral_rigidity.py: Sigma^2(L) from level positions (cumsum of unfolded gaps). tools/exp_spectral_rigidity.py: ('gue_matrix', {'gen': lambda: generate_gue_gaps(600), 'type': 'dist-GUE'}), tools/exp_spectral_rigidity.py: gue_theory = (2.0 / np.pi**2) * np.log(L_values) + 0.44 tools/exp_spectral_rigidity.py: unfolded = gaps / np.mean(gaps) tools/exp_spectral_rigidity.py: levels = np.concatenate([[0], np.cumsum(unfolded)]) tools/exp_spectral_rigidity.py: g_s = unfolded.copy() tools/exp_spectral_rigidity.py: 'sig2_gue_theory': gue_theory.tolist(), exec /bin/bash -lc "sed -n '260,560p' tools/exp_scale_selective_perturbation.py" in /opt/MM_D-ND succeeded in 0ms: # PCA across all (perturbation_type, alpha) profiles profile_matrix = np.array([p[2] for p in all_profiles]) # Center profile_mean = profile_matrix.mean(axis=0) profile_centered = profile_matrix - profile_mean U, S, Vt = np.linalg.svd(profile_centered, full_matrices=False) total_var = np.sum(S**2) explained = (S**2 / total_var) if total_var > 1e-15 else S * 0 # Effective rank (entropy-based) p_norm = explained[explained > 1e-12] p_norm = p_norm / p_norm.sum() eff_rank = float(np.exp(-np.sum(p_norm * np.log(p_norm)))) # Angle between perturbation-type centroids centroids = {} for pert_name in PERTURBATION_TYPES: vecs = [p[2] for p in all_profiles if p[0] == pert_name] centroids[pert_name] = np.mean(vecs, axis=0) # Pairwise cosine similarity between centroids cos_sim = {} pert_names_list = list(PERTURBATION_TYPES.keys()) for i in range(len(pert_names_list)): for j in range(i+1, len(pert_names_list)): a = centroids[pert_names_list[i]] b = centroids[pert_names_list[j]] na, nb = np.linalg.norm(a), np.linalg.norm(b) if na > 1e-12 and nb > 1e-12: cs = float(np.dot(a, b) / (na * nb)) else: cs = 0.0 cos_sim[f'{pert_names_list[i]}_vs_{pert_names_list[j]}'] = cs domain_result['pca'] = { 'singular_values': S.tolist(), 'explained_variance': explained.tolist(), 'effective_rank': eff_rank, 'PC1_loadings': Vt[0].tolist() if len(Vt) > 0 else [], 'PC2_loadings': Vt[1].tolist() if len(Vt) > 1 else [], } domain_result['centroid_cosine_similarity'] = cos_sim print(f"PCA explained: {explained[:3]}") print(f"Effective rank: {eff_rank:.3f}") print(f"Centroid cosine similarities: {cos_sim}") # Print retention table per perturbation type at alpha=0.5 print(f"\nRetention at alpha=0.5:") print(f"{'Perturbation':<20} {'SR':>8} {'L1':>8} {'L2':>8} {'SR2':>8} {'triple':>8}") for pert_name in PERTURBATION_TYPES: ret = domain_result['perturbations'][pert_name]['alpha_0.5']['retention'] print(f"{pert_name:<20} {ret['SR']:>8.3f} {ret['L1']:>8.3f} {ret['L2']:>8.3f} {ret['SR2']:>8.3f} {ret['triple_var']:>8.3f}") results[domain_name] = domain_result return results if __name__ == '__main__': parser = argparse.ArgumentParser() parser.add_argument('--N', type=int, default=30000) parser.add_argument('--seed', type=int, default=20260506) parser.add_argument('--trials', type=int, default=16) args = parser.parse_args() results = run_experiment(N=args.N, seed=args.seed, n_trials=args.trials) outpath = Path('tools/data/scale_selective_perturbation.json') with open(outpath, 'w') as f: json.dump(results, f, indent=2, default=str) print(f"\nSaved to {outpath}") codex Creo uno strumento riusabile che misura la “dimensionalità di perturbazione” su repliche indipendenti. L’obiettivo è separare tre possibilità: asse GUE stabile, artefatto da singola matrice piccola, oppure effetto generico di campioni corti. apply patch patch: completed /opt/MM_D-ND/tools/exp_perturbation_dimensionality_audit.py diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..a3142f12cff110385e3f1d5cef3b315bba5e5ac7 --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..a3142f12cff110385e3f1d5cef3b315bba5e5ac7 --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + run(args) + + +if __name__ == "__main__": + main() codex Lo strumento è pronto. Lo eseguo con perimetro moderato: primi e controlli a 12k gap, GUE con 6 repliche indipendenti da 16 matrici 180x180, più GUE corto come controllo di campione piccolo. exec /bin/bash -lc 'python tools/exp_perturbation_dimensionality_audit.py' in /opt/MM_D-ND diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..a3142f12cff110385e3f1d5cef3b315bba5e5ac7 --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + run(args) + + +if __name__ == "__main__": + main() succeeded in 1283ms: fixed domains domain N rank PC2 cos(adj,large) primes 12000 1.374 0.070 0.947 prime_shuffle_control 12000 2.294 0.199 0.247 poisson 12000 1.917 0.193 0.918 GUE independent replicates gue_rep_0 2288 1.651 0.128 0.957 gue_rep_1 2288 1.131 0.024 0.931 gue_rep_2 2288 1.062 0.008 0.725 gue_rep_3 2288 1.283 0.057 0.869 gue_rep_4 2288 1.056 0.007 0.902 gue_rep_5 2288 1.645 0.163 0.875 summary gue rank=1.305+/-0.278 PC2=0.064+/-0.066 cos=0.877+/-0.081 gue_short rank=1.683+/-0.498 PC2=0.106+/-0.080 cos=0.567+/-0.340 saved tools/data/perturbation_dimensionality_audit.json diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..a3142f12cff110385e3f1d5cef3b315bba5e5ac7 --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + run(args) + + +if __name__ == "__main__": + main() codex Il primo output non conferma il secondo asse GUE forte: sulle repliche lunghe GUE la media è rank 1.305, PC2 6.4%, molto sotto il 25.2% del run precedente. Prima di chiudere controllo un punto META importante: gli stessi nomi osservabili potrebbero non indicare le stesse funzioni tra gli script. exec /bin/bash -lc 'rg -n "def .*spacing|def .*spectral|def .*sr|OBSERVABLE|SR2|triple" tools/exp_scale_selective_perturbation.py tools/exp_observable_rank_audit.py tools/exp_3d_boundary_layers.py tools/exp_selective_layer_decoupling.py' in /opt/MM_D-ND succeeded in 0ms: tools/exp_selective_layer_decoupling.py:5:Consecutio from BOUNDARY (piano 60h): The two Markov layers (pairs->SR,L1; triples->SR2,L2) tools/exp_selective_layer_decoupling.py:16: (preserves pair statistics at alpha=1, destroys triple+) tools/exp_selective_layer_decoupling.py:18: (preserves triple statistics at alpha=1) tools/exp_selective_layer_decoupling.py:55:def gen_gue_spacings(N, rng): tools/exp_selective_layer_decoupling.py:66:def gen_poisson_spacings(N, rng): tools/exp_selective_layer_decoupling.py:130:def spacing_ratio(gaps): tools/exp_selective_layer_decoupling.py:145:def next_nearest_sr(gaps): tools/exp_selective_layer_decoupling.py:177:def triple_var(gaps): tools/exp_selective_layer_decoupling.py:184:OBSERVABLES = { tools/exp_selective_layer_decoupling.py:187: 'SR2': next_nearest_sr, tools/exp_selective_layer_decoupling.py:190: 'triple_var': triple_var, tools/exp_selective_layer_decoupling.py:194:LAYER2_OBS = ['SR2', 'L2', 'cond_entropy', 'triple_var'] tools/exp_selective_layer_decoupling.py:198: return {name: fn(gaps) for name, fn in OBSERVABLES.items()} tools/exp_selective_layer_decoupling.py:219: results = {name: [] for name in OBSERVABLES} tools/exp_selective_layer_decoupling.py:222: surr_obs = {name: [] for name in OBSERVABLES} tools/exp_selective_layer_decoupling.py:237: for name in OBSERVABLES: tools/exp_selective_layer_decoupling.py:240: for name in OBSERVABLES: tools/exp_selective_layer_decoupling.py:283: for name in OBSERVABLES: tools/exp_selective_layer_decoupling.py:300: for name in OBSERVABLES: tools/exp_scale_selective_perturbation.py:6:observables (SR, L1, L2, SR2, triple_var) collapse to 1 latent coordinate tools/exp_scale_selective_perturbation.py:16:For each perturbation at multiple intensities, measure SR/L1/L2/SR2/triple_var, tools/exp_scale_selective_perturbation.py:72:def spectral_rigidity(gaps, L=10): tools/exp_scale_selective_perturbation.py:108:def sr2(gaps, L=20): tools/exp_scale_selective_perturbation.py:113:def triple_variance(gaps): tools/exp_scale_selective_perturbation.py:114: """Variance of triple products g_n * g_{n+1} * g_{n+2}.""" tools/exp_scale_selective_perturbation.py:115: triples = gaps[:-2] * gaps[1:-1] * gaps[2:] tools/exp_scale_selective_perturbation.py:116: return float(np.var(triples)) tools/exp_scale_selective_perturbation.py:125: 'SR2': sr2(gaps), tools/exp_scale_selective_perturbation.py:126: 'triple_var': triple_variance(gaps), tools/exp_scale_selective_perturbation.py:192: obs_names = ['SR', 'L1', 'L2', 'SR2', 'triple_var'] tools/exp_scale_selective_perturbation.py:309: print(f"{'Perturbation':<20} {'SR':>8} {'L1':>8} {'L2':>8} {'SR2':>8} {'triple':>8}") tools/exp_scale_selective_perturbation.py:312: print(f"{pert_name:<20} {ret['SR']:>8.3f} {ret['L1']:>8.3f} {ret['L2']:>8.3f} {ret['SR2']:>8.3f} {ret['triple_var']:>8.3f}") tools/exp_3d_boundary_layers.py:7: - Layer 2 (triples, Mk2) → SR2, triple_var (depth) tools/exp_3d_boundary_layers.py:63:def obs_spacing_ratio(gaps): tools/exp_3d_boundary_layers.py:73:# --- Layer 2 observables (triple statistics) --- tools/exp_3d_boundary_layers.py:74:def obs_sr2(gaps): tools/exp_3d_boundary_layers.py:79:def obs_triple_var(gaps): tools/exp_3d_boundary_layers.py:80: """Variance of consecutive triple sums, normalized.""" tools/exp_3d_boundary_layers.py:81: triples = gaps[:-2] + gaps[1:-1] + gaps[2:] tools/exp_3d_boundary_layers.py:84: return np.var(triples) / v tools/exp_3d_boundary_layers.py:92: 'SR2': obs_sr2, tools/exp_3d_boundary_layers.py:93: 'triple_var': obs_triple_var, tools/exp_3d_boundary_layers.py:198: obs_names = ['SR', 'L1', 'SR2', 'triple_var'] tools/exp_3d_boundary_layers.py:199: layer_map = {'SR': 'L1_pair', 'L1': 'L1_pair', 'SR2': 'L2_triple', 'triple_var': 'L2_triple'} tools/exp_3d_boundary_layers.py:203: 'question': 'Do Layer 1 (pairs) and Layer 2 (triples) transition at the same critical alpha?', tools/exp_3d_boundary_layers.py:232: l2_crit = np.mean([seq_data['critical_alpha_50'][o] for o in ['SR2', 'triple_var']]) tools/exp_observable_rank_audit.py:27:def obs_spacing_ratio(gaps): tools/exp_observable_rank_audit.py:41:def obs_sr2(gaps): tools/exp_observable_rank_audit.py:47:def obs_triple_var(gaps): tools/exp_observable_rank_audit.py:48: triples = gaps[:-2] + gaps[1:-1] + gaps[2:] tools/exp_observable_rank_audit.py:52: return float(np.var(triples) / v) tools/exp_observable_rank_audit.py:55:OBSERVABLES = { tools/exp_observable_rank_audit.py:59: "SR2": obs_sr2, tools/exp_observable_rank_audit.py:60: "triple_var": obs_triple_var, tools/exp_observable_rank_audit.py:65: return {name: fn(gaps) for name, fn in OBSERVABLES.items()} tools/exp_observable_rank_audit.py:69: vals = {name: [] for name in OBSERVABLES} tools/exp_observable_rank_audit.py:87: vals = {name: [] for name in OBSERVABLES} tools/exp_observable_rank_audit.py:95: for name in OBSERVABLES: tools/exp_observable_rank_audit.py:109: names = list(OBSERVABLES) tools/exp_observable_rank_audit.py:145: for obs_name in OBSERVABLES: tools/exp_observable_rank_audit.py:193: print(f"{'sequence':<10} {'pc1':>8} {'eff_rank':>9} {'mean|corr|':>11} z(SR,L1,L2,SR2,triple)") tools/exp_observable_rank_audit.py:200: z_text = ", ".join(f"{obs}={z[obs]:+.1f}" for obs in OBSERVABLES) exec /bin/bash -lc "python - <<'PY' import json p='tools/data/perturbation_dimensionality_audit.json' with open(p) as f: d=json.load(f) for name,res in d['fixed_domains'].items(): print(name, res['pca']['effective_rank'], res['pca']['explained_variance'][:3], res['original_vs_shuffle_z']) print('gue_summary', d['gue_summary']) print('gue_short_summary', d['gue_short_summary']) PY" in /opt/MM_D-ND succeeded in 0ms: primes 1.3742663637803922 [0.9190556645881014, 0.070101017006854, 0.008983425770601017] {'SR': -11.771588798796008, 'L1': -6.812661387466671, 'L2': -2.0251197079759917, 'SR2': -2.6089661657848535, 'triple_var': -6.521692785375022} prime_shuffle_control 2.2940584424934354 [0.7085642282320445, 0.19866880899045572, 0.07553546190455197] {'SR': 0.19864198788770907, 'L1': 0.18875620635920462, 'L2': -2.1221272594320055, 'SR2': -0.8010044893735694, 'triple_var': -0.2735216058086641} poisson 1.9170970847698845 [0.7725501335941172, 0.19334250403644737, 0.02618884142162428] {'SR': -1.059345493450584, 'L1': -0.9841031737027609, 'L2': 0.24839105672664213, 'SR2': 0.557113236857107, 'triple_var': -0.8642253770968523} gue_summary {'n_replicates': 6, 'effective_rank_mean': 1.3046902198293868, 'effective_rank_std': 0.27834375275528955, 'effective_rank_min': 1.0559542042494687, 'effective_rank_max': 1.6510620473338578, 'pc2_mean': 0.06437882111028331, 'pc2_std': 0.06628602964141048, 'adjacent_vs_large_cosine_mean': 0.8765816067186908, 'adjacent_vs_large_cosine_std': 0.08114845671432144} gue_short_summary {'n_replicates': 6, 'effective_rank_mean': 1.6827632602096403, 'effective_rank_std': 0.49843227861145134, 'effective_rank_min': 1.1102802735943371, 'effective_rank_max': 2.3344850607290666, 'pc2_mean': 0.10572237803352125, 'pc2_std': 0.08035031619191095, 'adjacent_vs_large_cosine_mean': 0.5667749702300134, 'adjacent_vs_large_cosine_std': 0.33974934276048974} diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..a3142f12cff110385e3f1d5cef3b315bba5e5ac7 --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + run(args) + + +if __name__ == "__main__": + main() codex C’è un fatto META concreto: `SR` e `SR2` non sono semanticamente stabili tra gli script. Estendo lo strumento con due set osservabili espliciti, così posso distinguere “rank-audit set” e “scale-0330 set” senza far decidere ai nomi. apply patch patch: completed /opt/MM_D-ND/tools/exp_perturbation_dimensionality_audit.py diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() codex Ora rieseguo il controllo con il set osservabile del run 03:30, salvando un secondo JSON. Questo rende il confronto diretto sul claim più forte, lasciando il primo JSON come controllo sul set rank-audit. exec /bin/bash -lc 'python tools/exp_perturbation_dimensionality_audit.py --observable-set scale_0330 --out tools/data/perturbation_dimensionality_audit_scale0330.json' in /opt/MM_D-ND diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() codex Il run con rigidità spettrale è più lento, come previsto, perché ogni profilo fa fit locali. Lo lascio completare: serve per evitare un report che confonda cambio di osservabile con cambio di fenomeno. diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() succeeded in 118235ms: fixed domains domain N rank PC2 cos(adj,large) primes 12000 1.318 0.046 0.975 prime_shuffle_control 12000 1.988 0.085 0.526 poisson 12000 2.201 0.198 0.885 GUE independent replicates gue_rep_0 2288 1.403 0.092 0.968 gue_rep_1 2288 1.124 0.024 0.935 gue_rep_2 2288 1.142 0.028 0.796 gue_rep_3 2288 1.364 0.090 0.848 gue_rep_4 2288 1.580 0.164 0.768 gue_rep_5 2288 1.672 0.194 0.927 summary gue rank=1.381+/-0.223 PC2=0.099+/-0.069 cos=0.874+/-0.082 gue_short rank=2.013+/-0.525 PC2=0.159+/-0.087 cos=0.746+/-0.242 saved tools/data/perturbation_dimensionality_audit_scale0330.json exec /bin/bash -lc 'git status --short' in /opt/MM_D-ND succeeded in 0ms: M applications/INDEX.draft.json M applications/soluzioni/20260430_0330_three-regimes-of-modular-memory-algebraic/finding_index.draft.json M applications/soluzioni/20260430_0330_three-regimes-of-modular-memory-algebraic/manifest.draft.json M tools/data/agent_field_live.md M tools/data/conoscenza_teorie.json M tools/data/consecutio.json M tools/data/consecutio_processata.json M tools/data/domande_fondamentali.json M tools/data/incrocio_risultato.json M tools/data/lab_data.json M tools/data/lab_graph.json M tools/data/lab_health.json M tools/data/lab_session_log.jsonl M tools/data/pipeline_state.json M tools/data/ponti_evoluti.json M tools/data/refresh_detector_state.json M tools/data/seme.json D tools/data/seme_archive/piano_37.json D tools/data/seme_archive/piano_38.json D tools/data/seme_archive/piano_4.json D tools/data/seme_archive/piano_5.json D tools/data/seme_archive/piano_6.json M tools/data/seme_axioms.json M tools/data/seme_backup_pre_run.json M tools/data/tm1_figures/tensions.json M tools/data/tm1_figures/tensions_raw.json ?? applications/published/20260504_0901_the-two-markov-layers-are-coupled/ ?? applications/published/20260504_1219_markov-layers-pass-the-first-recovery/ ?? applications/published/20260505_0330_observable-rank-audit-many-probes-one/ ?? applications/scoperte/20260502_0330_the-dipolar-phase-transition-is-nearly_auto/ ?? applications/scoperte/20260503_0330_markov-memory-has-two-visible-layers_auto/ ?? applications/scoperte/20260504_0901_the-two-markov-layers-are-coupled_auto/ ?? applications/scoperte/20260504_1219_markov-layers-pass-the-first-recovery_auto/ ?? applications/scoperte/20260505_0330_observable-rank-audit-many-probes-one_auto/ ?? applications/soluzioni/20260502_0330_the-dipolar-phase-transition-is-nearly/ ?? applications/soluzioni/20260503_0330_markov-memory-has-two-visible-layers/ ?? applications/soluzioni/20260504_0901_the-two-markov-layers-are-coupled/ ?? applications/soluzioni/20260504_1219_markov-layers-pass-the-first-recovery/ ?? applications/soluzioni/20260505_0330_observable-rank-audit-many-probes-one/ ?? tools/data/.last_telegram_msg ?? tools/data/3d_boundary_layers.json ?? tools/data/aeternitas/ ?? tools/data/alignment_active.json ?? tools/data/alignment_markers.jsonl ?? tools/data/bicono_projections.jsonl ?? tools/data/boundary_coherence.json ?? tools/data/boundary_shuffle_audit.json ?? tools/data/brody_calibration_results.json ?? tools/data/brody_flow.json ?? tools/data/conoscenza_teorie.json.bak.retraction_22_04 ?? tools/data/cross_domain_dipolar_direction.json ?? tools/data/cross_observable_consistency.json ?? tools/data/crossover_phase_test.json ?? tools/data/dipolar_crossover.json ?? tools/data/dipolar_vector_scaling.json ?? tools/data/domandatore/domandatore_20260421_0746.json ?? tools/data/domandatore/domandatore_20260422_0345.json ?? tools/data/domandatore/domandatore_20260423_0345.json ?? tools/data/domandatore/domandatore_20260424_0345.json ?? tools/data/domandatore/domandatore_20260425_0345.json ?? tools/data/domandatore/domandatore_20260426_0345.json ?? tools/data/domandatore/domandatore_20260427_0345.json ?? tools/data/domandatore/domandatore_20260428_0345.json ?? tools/data/domandatore/domandatore_20260428_1236.json ?? tools/data/domandatore/domandatore_20260429_0345.json ?? tools/data/domandatore/domandatore_20260430_0345.json ?? tools/data/domandatore/domandatore_20260501_0345.json ?? tools/data/domandatore/domandatore_20260502_0345.json ?? tools/data/domandatore/domandatore_20260503_0345.json ?? tools/data/domandatore/domandatore_20260504_0345.json ?? tools/data/domandatore/domandatore_20260505_0345.json ?? tools/data/domandatore/domandatore_20260506_0345.json ?? tools/data/evolution/evolution_20260422_0330.md ?? tools/data/evolution/evolution_20260422_1616.md ?? tools/data/evolution/evolution_20260423_0330.md ?? tools/data/evolution/evolution_20260424_0330.md ?? tools/data/evolution/evolution_20260425_0330.md ?? tools/data/evolution/evolution_20260427_0330.md ?? tools/data/evolution/evolution_20260428_0330.md ?? tools/data/evolution/evolution_20260429_0852.md ?? tools/data/evolution/evolution_20260429_1013.md ?? tools/data/evolution/evolution_20260430_0330.md ?? tools/data/evolution/evolution_20260430_1905.md ?? tools/data/evolution/evolution_20260430_1946.md ?? tools/data/evolution/evolution_20260501_0330.md ?? tools/data/evolution/evolution_20260501_0725.md ?? tools/data/evolution/evolution_20260501_0931.md ?? tools/data/evolution/evolution_20260502_0330.md ?? tools/data/evolution/evolution_20260503_0330.md ?? tools/data/evolution/evolution_20260504_0901.md ?? tools/data/evolution/evolution_20260504_1219.md ?? tools/data/evolution/evolution_20260505_0330.md ?? tools/data/evolution/evolution_20260506_0330.md ?? tools/data/incrocio_20260422_0336.json ?? tools/data/incrocio_20260423_0335.json ?? tools/data/incrocio_20260424_0347.json ?? tools/data/incrocio_20260425_0339.json ?? tools/data/incrocio_20260428_0340.json ?? tools/data/incrocio_20260429_0859.json ?? tools/data/lab_bridge_issues.jsonl ?? tools/data/magnitude_psd_from_acf.json ?? tools/data/markov3_observable_hunt.json ?? tools/data/markov_dipolar_decomposition.json ?? tools/data/markov_k_direction.json ?? tools/data/markov_layer_recovery_audit.json ?? tools/data/markov_memory_by_gue_type.json ?? tools/data/markov_scale_function.json ?? tools/data/meta_tautology_test.json ?? tools/data/mod3_scaling.json ?? tools/data/mod3_vs_residual_ordering.json ?? tools/data/modular_algebra_depth.json ?? tools/data/modular_memory_spectrum.json ?? tools/data/observable_rank_audit.json ?? tools/data/observable_rank_audit_seed20260506.json ?? tools/data/perturbation_dimensionality_audit.json ?? tools/data/perturbation_dimensionality_audit_scale0330.json ?? tools/data/reports/_quarantine_falsifier_29_04/ ?? tools/data/reports/agent_20260422_0330.md ?? tools/data/reports/agent_20260422_1616.md ?? tools/data/reports/agent_20260423_0330.md ?? tools/data/reports/agent_20260424_0330.md ?? tools/data/reports/agent_20260425_0330.md ?? tools/data/reports/agent_20260426_0330.md ?? tools/data/reports/agent_20260427_0330.md ?? tools/data/reports/agent_20260428_0330.md ?? tools/data/reports/agent_20260429_1013.md ?? tools/data/reports/agent_20260429_1041.md ?? tools/data/reports/agent_20260430_0330.md ?? tools/data/reports/agent_20260430_1905.md ?? tools/data/reports/agent_20260430_1919.md ?? tools/data/reports/agent_20260430_1946.md ?? tools/data/reports/agent_20260501_0330.md ?? tools/data/reports/agent_20260501_0725.md ?? tools/data/reports/agent_20260501_0931.md ?? tools/data/reports/agent_20260502_0330.md ?? tools/data/reports/agent_20260503_0330.md ?? tools/data/reports/agent_20260504_0901.md ?? tools/data/reports/agent_20260504_1219.md ?? tools/data/reports/agent_20260505_0330.md ?? tools/data/reports/agent_20260505_1022.md ?? tools/data/reports/agent_20260506_0330.md ?? tools/data/reports/ddf_20260422_0405.json ?? tools/data/reports/ddf_20260423_0405.json ?? tools/data/reports/ddf_20260424_0405.json ?? tools/data/reports/ddf_20260425_0405.json ?? tools/data/reports/ddf_20260426_0405.json ?? tools/data/reports/ddf_20260427_0405.json ?? tools/data/reports/ddf_20260428_0405.json ?? tools/data/reports/ddf_20260429_0405.json ?? tools/data/reports/ddf_20260430_0405.json ?? tools/data/reports/ddf_20260501_0405.json ?? tools/data/reports/ddf_20260502_0405.json ?? tools/data/reports/ddf_20260503_0405.json ?? tools/data/reports/ddf_20260504_0405.json ?? tools/data/reports/ddf_20260505_0405.json ?? tools/data/reports/ddf_20260505_0636.json ?? tools/data/reports/ddf_20260506_0405.json ?? tools/data/reports/evolution_20260422_0330.md ?? tools/data/reports/evolution_20260422_1616.md ?? tools/data/reports/evolution_20260423_0330.md ?? tools/data/reports/evolution_20260424_0330.md ?? tools/data/reports/evolution_20260425_0330.md ?? tools/data/reports/evolution_20260427_0330.md ?? tools/data/reports/evolution_20260428_0330.md ?? tools/data/reports/evolution_20260503_0330.md ?? tools/data/reports/evolution_20260504_0330.md ?? tools/data/reports/evolution_20260505_0330.md ?? tools/data/reports/evolution_20260506_0330.md ?? tools/data/reports/falsifier_20260429_1013.json ?? tools/data/reports/falsifier_20260429_1041.json ?? tools/data/reports/falsifier_20260430_0330.json ?? tools/data/reports/falsifier_20260430_1905.json ?? tools/data/reports/falsifier_20260430_1919.json ?? tools/data/reports/falsifier_20260430_1946.json ?? tools/data/reports/falsifier_20260501_0330.json ?? tools/data/reports/falsifier_20260501_0725.json ?? tools/data/reports/falsifier_20260501_0931.json ?? tools/data/reports/falsifier_20260502_0330.json ?? tools/data/reports/falsifier_20260503_0330.json ?? tools/data/reports/falsifier_20260504_0901.json ?? tools/data/reports/falsifier_20260504_1219.json ?? tools/data/reports/falsifier_20260505_0330.json ?? tools/data/reports/falsifier_20260506_0330.raw.txt ?? tools/data/reports/incident_20260504_0721.md ?? tools/data/reports/incident_20260504_1138.md ?? tools/data/reports/latest.md ?? tools/data/scale_selective_perturbation.json ?? tools/data/selective_layer_decoupling.json ?? tools/data/seme_archive/piano_60.json ?? tools/data/seme_archive/piano_61.json ?? tools/data/seme_archive/piano_62.json ?? tools/data/seme_archive/piano_63.json ?? tools/data/seme_archive/piano_64.json ?? tools/data/spectral_rigidity_results.json ?? tools/data/two_channel_boundary.json ?? tools/data/two_channel_cross_domain.json ?? tools/data/two_channel_shuffle_audit.json ?? tools/data/two_layer_universality.json ?? tools/data/valutatore_log.jsonl ?? tools/data/veritas/ ?? tools/exp_3d_boundary_layers.py ?? tools/exp_boundary_coherence.py ?? tools/exp_boundary_shuffle_audit.py ?? tools/exp_brody_calibration.py ?? tools/exp_brody_flow.py ?? tools/exp_cross_domain_dipolar_direction.py ?? tools/exp_cross_observable_consistency.py ?? tools/exp_crossover_phase_test.py ?? tools/exp_dipolar_angle_reference.py ?? tools/exp_dipolar_crossover.py ?? tools/exp_dipolar_vector_scaling.py ?? tools/exp_magnitude_psd_from_acf.py ?? tools/exp_markov3_observable_hunt.py ?? tools/exp_markov_dipolar_decomposition.py ?? tools/exp_markov_k_direction.py ?? tools/exp_markov_layer_recovery_audit.py ?? tools/exp_markov_memory_by_gue_type.py ?? tools/exp_markov_psd_prediction.py ?? tools/exp_markov_scale_function.py ?? tools/exp_meta_tautology_test.py ?? tools/exp_mod3_scaling.py ?? tools/exp_mod3_vs_residual_ordering.py ?? tools/exp_modular_algebra_depth.py ?? tools/exp_modular_memory_spectrum.py ?? tools/exp_observable_rank_audit.py ?? tools/exp_perturbation_dimensionality_audit.py ?? tools/exp_scale_selective_perturbation.py ?? tools/exp_selective_layer_decoupling.py ?? tools/exp_spectral_rigidity.py ?? tools/exp_two_channel_boundary.py ?? tools/exp_two_channel_cross_domain.py ?? tools/exp_two_channel_shuffle_audit.py ?? tools/exp_two_layer_universality.py exec /bin/bash -lc "python - <<'PY' import json for p in ['tools/data/perturbation_dimensionality_audit.json','tools/data/perturbation_dimensionality_audit_scale0330.json']: with open(p) as f: d=json.load(f) print('\\n', p, d['params']['observable_set']) for name,res in d['fixed_domains'].items(): pc2=res['pca']['explained_variance'][1] cos=res['pca']['centroid_cosine']['adjacent_swap_vs_large_gap_only'] print(f\"{name:22s} N={res['n_gaps']:5d} rank={res['pca']['effective_rank']:.3f} pc2={pc2:.3f} cos={cos:.3f}\") print('gue', d['gue_summary']) print('gue_short', d['gue_short_summary']) PY" in /opt/MM_D-ND exited 1 in 0ms: Traceback (most recent call last): File "", line 4, in KeyError: 'observable_set' diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() codex Il JSON rank-audit era stato prodotto prima dell’aggiunta del campo `observable_set`; lo rigenero per avere metadati coerenti nei due output. I risultati numerici possono oscillare solo per seed interno, ma il seed radice resta fissato. exec /bin/bash -lc 'python tools/exp_perturbation_dimensionality_audit.py --observable-set rank_audit --out tools/data/perturbation_dimensionality_audit.json' in /opt/MM_D-ND diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() succeeded in 1169ms: fixed domains domain N rank PC2 cos(adj,large) primes 12000 1.374 0.070 0.947 prime_shuffle_control 12000 2.294 0.199 0.247 poisson 12000 1.917 0.193 0.918 GUE independent replicates gue_rep_0 2288 1.651 0.128 0.957 gue_rep_1 2288 1.131 0.024 0.931 gue_rep_2 2288 1.062 0.008 0.725 gue_rep_3 2288 1.283 0.057 0.869 gue_rep_4 2288 1.056 0.007 0.902 gue_rep_5 2288 1.645 0.163 0.875 summary gue rank=1.305+/-0.278 PC2=0.064+/-0.066 cos=0.877+/-0.081 gue_short rank=1.683+/-0.498 PC2=0.106+/-0.080 cos=0.567+/-0.340 saved tools/data/perturbation_dimensionality_audit.json diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() exec /bin/bash -lc "python - <<'PY' import json for p in ['tools/data/perturbation_dimensionality_audit.json','tools/data/perturbation_dimensionality_audit_scale0330.json']: with open(p) as f: d=json.load(f) print('\\n', p, d['params']['observable_set']) for name,res in d['fixed_domains'].items(): pc2=res['pca']['explained_variance'][1] cos=res['pca']['centroid_cosine']['adjacent_swap_vs_large_gap_only'] print(f\"{name:22s} N={res['n_gaps']:5d} rank={res['pca']['effective_rank']:.3f} pc2={pc2:.3f} cos={cos:.3f}\") print('gue_rank_mean', d['gue_summary']['effective_rank_mean'], 'pc2_mean', d['gue_summary']['pc2_mean'], 'short_rank_mean', d['gue_short_summary']['effective_rank_mean'], 'short_pc2_mean', d['gue_short_summary']['pc2_mean']) PY" in /opt/MM_D-ND succeeded in 0ms: tools/data/perturbation_dimensionality_audit.json rank_audit primes N=12000 rank=1.374 pc2=0.070 cos=0.947 prime_shuffle_control N=12000 rank=2.294 pc2=0.199 cos=0.247 poisson N=12000 rank=1.917 pc2=0.193 cos=0.918 gue_rank_mean 1.3046902198293868 pc2_mean 0.06437882111028331 short_rank_mean 1.6827632602096403 short_pc2_mean 0.10572237803352125 tools/data/perturbation_dimensionality_audit_scale0330.json scale_0330 primes N=12000 rank=1.318 pc2=0.046 cos=0.975 prime_shuffle_control N=12000 rank=1.988 pc2=0.085 cos=0.526 poisson N=12000 rank=2.201 pc2=0.198 cos=0.885 gue_rank_mean 1.3807945090013878 pc2_mean 0.09883354928222045 short_rank_mean 2.0130700509856583 short_pc2_mean 0.15886129521118472 diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() codex Ho i dati finali. Scrivo il report nel file richiesto e poi aggiorno il seme con il vincolo emerso, tenendo separati finding numerico e correzione metodologica. apply patch patch: completed /opt/MM_D-ND/tools/data/reports/agent_20260506_0625.md diff --git a/tools/data/reports/agent_20260506_0625.md b/tools/data/reports/agent_20260506_0625.md new file mode 100644 index 0000000000000000000000000000000000000000..5769f28992639f3558821f09898516122050563e --- /dev/null +++ b/tools/data/reports/agent_20260506_0625.md @@ -0,0 +1,80 @@ +# Agent Report — Perturbation Dimensionality Is Not Yet a Stable GUE Invariant +**Date**: 2026-05-06 06:25 +**Piano**: 65 +**Tension explored**: META + BOUNDARY + +## Claim Under Test +The 03:30 run reported that scale-selective perturbations reveal a second axis in GUE (effective rank 1.889, PC2=25.2%) but not in primes. The caveat was explicit: the GUE sample was short (`N=253`). This run tests whether that second axis survives independent GUE ensembles and whether it depends on the observable definitions hidden under the same labels (`SR`, `SR2`, `triple_var`). + +## Experiment +Tool created: `tools/exp_perturbation_dimensionality_audit.py` + +Perimeter atomico: +- fixed domains: `primes` 12,000 gaps, `prime_shuffle_control` 12,000 permuted prime gaps, `poisson` 12,000 iid exponential spacings; +- GUE long control: 6 independent replicates, each from 16 Hermitian matrices of size 180, after 10% edge trim; each replicate has 2,288 spacings; +- GUE short control: 6 independent one-matrix replicates of size 42, after edge trim; this approximates the fragile small-sample regime; +- perturbations: `adjacent_swap`, `block_shuffle`, `large_gap_only`, `uniform`; +- alpha grid: 0.1, 0.3, 0.5, 0.7, 0.9; +- trials per perturbation-alpha: 10; full-shuffle baseline: 24; +- seed: 20260506. + +Two observable sets were run because a META issue emerged: +- `rank_audit`: `SR` = nearest-neighbor spacing ratio, `SR2` = next-nearest spacing ratio, `triple_var` = normalized variance of triple sums. +- `scale_0330`: `SR` = local spectral rigidity at L=10, `SR2` = local spectral rigidity at L=20, `triple_var` = variance of triple products. + +## Results + +### Rank-audit observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.374 | 0.070 | 0.947 | +| prime_shuffle_control | 12000 | 2.294 | 0.199 | 0.247 | +| poisson | 12000 | 1.917 | 0.193 | 0.918 | +| GUE long, 6 reps mean | 2288 each | 1.305 ± 0.278 | 0.064 ± 0.066 | 0.877 ± 0.081 | +| GUE short, 6 reps mean | short | 1.683 ± 0.498 | 0.106 ± 0.080 | 0.567 ± 0.340 | + +### Scale-0330 observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.318 | 0.046 | 0.975 | +| prime_shuffle_control | 12000 | 1.988 | 0.085 | 0.526 | +| poisson | 12000 | 2.201 | 0.198 | 0.885 | +| GUE long, 6 reps mean | 2288 each | 1.381 ± 0.223 | 0.099 ± 0.069 | 0.874 ± 0.082 | +| GUE short, 6 reps mean | short | 2.013 ± 0.525 | 0.159 ± 0.087 | 0.746 ± 0.242 | + +## Findings + +1. **The strong GUE second-axis claim does not survive as stated.** Under direct `scale_0330` observables, long independent GUE replicates give rank 1.381 ± 0.223 and PC2 9.9% ± 6.9%, not rank 1.889 and PC2 25.2%. The previous number is inside the fragile short-sample regime: GUE short controls have rank 2.013 ± 0.525 and PC2 15.9% ± 8.7%. + +2. **Short GUE samples inflate apparent perturbation dimensionality.** In both observable sets, GUE short has higher rank and larger variance than GUE long. This does not prove the 03:30 axis was false in every configuration; it restricts it to a sample-size-sensitive observation unless a larger-replicate run recovers it. + +3. **The lab has an observable-name collision.** `SR`, `SR2`, and `triple_var` do not name the same functions across the recent scripts. `exp_observable_rank_audit.py` uses spacing-ratio and triple-sum variance; `exp_scale_selective_perturbation.py` uses local spectral rigidity and triple-product variance. Therefore the sentence "same observables as observable rank audit" in the 03:30 report is not exact. This is a META constraint, not a numerical subtlety. + +4. **Primes remain close to one perturbation coordinate in both observable sets.** Primes rank is 1.374 in `rank_audit` and 1.318 in `scale_0330`; PC2 is 7.0% and 4.6%. This part of the 03:30 asymmetry is stable in the tested perimeter. + +5. **Poisson and shuffled-prime controls show multi-axis artifacts.** Poisson has rank 1.917/2.201 depending on observable set; prime shuffle control has rank 2.294/1.988. Multi-dimensional perturbation response by itself is not evidence of structured GUE-like boundary. It can arise from low structural signal plus noisy denominators in retention normalization. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: "GUE has a second perturbation axis" must be scoped to the exact sample length, generator, and observable definitions. In the larger independent-GUE perimeter tested here, the robust statement is weaker: + +> Primes remain near one perturbation coordinate under both observable sets; GUE long replicates show only a weak second component; short GUE samples can inflate apparent rank; Poisson and shuffled controls can also appear multi-axis. + +The boundary is still operator-dependent, but perturbation dimensionality is not yet a stable domain invariant. The next valid test is not another single GUE matrix; it is a replicate-and-size curve for effective rank vs number of spacings, with observable definitions versioned. + +## Consecutio +What opens now: the lab needs an observable registry. Labels like `SR` cannot travel between reports unless they bind to a function definition. Without that, META flags are not philosophical: the same label can silently change the object under test. + +## Auto-audit: 5 lenti +- **L1 hard constraint vs bias**: no zero/always claim. "Does not survive as stated" means the reported magnitude did not reproduce in the stated larger perimeter. +- **L2 quantity vs ratio**: effective rank and PC2 are reported with sample size and replicate variance. Retention ratios are not treated as absolute structure without controls. +- **L3 no silent patching**: the 03:30 claim is explicitly restricted, not renamed as confirmed. The observable-name mismatch is declared. +- **L4 edge cases**: the short-GUE effect is isolated as its own control, not averaged into long GUE. +- **L5 re-discovery**: PCA/effective-rank instability under small samples is a known statistical issue. This report is a lab constraint on framing, not a NEW mathematical result. + +## Files +- Script: `tools/exp_perturbation_dimensionality_audit.py` +- Data: `tools/data/perturbation_dimensionality_audit.json` +- Data: `tools/data/perturbation_dimensionality_audit_scale0330.json` +- Report: `tools/data/reports/agent_20260506_0625.md` diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() diff --git a/tools/data/reports/agent_20260506_0625.md b/tools/data/reports/agent_20260506_0625.md new file mode 100644 index 0000000000000000000000000000000000000000..5769f28992639f3558821f09898516122050563e --- /dev/null +++ b/tools/data/reports/agent_20260506_0625.md @@ -0,0 +1,80 @@ +# Agent Report — Perturbation Dimensionality Is Not Yet a Stable GUE Invariant +**Date**: 2026-05-06 06:25 +**Piano**: 65 +**Tension explored**: META + BOUNDARY + +## Claim Under Test +The 03:30 run reported that scale-selective perturbations reveal a second axis in GUE (effective rank 1.889, PC2=25.2%) but not in primes. The caveat was explicit: the GUE sample was short (`N=253`). This run tests whether that second axis survives independent GUE ensembles and whether it depends on the observable definitions hidden under the same labels (`SR`, `SR2`, `triple_var`). + +## Experiment +Tool created: `tools/exp_perturbation_dimensionality_audit.py` + +Perimeter atomico: +- fixed domains: `primes` 12,000 gaps, `prime_shuffle_control` 12,000 permuted prime gaps, `poisson` 12,000 iid exponential spacings; +- GUE long control: 6 independent replicates, each from 16 Hermitian matrices of size 180, after 10% edge trim; each replicate has 2,288 spacings; +- GUE short control: 6 independent one-matrix replicates of size 42, after edge trim; this approximates the fragile small-sample regime; +- perturbations: `adjacent_swap`, `block_shuffle`, `large_gap_only`, `uniform`; +- alpha grid: 0.1, 0.3, 0.5, 0.7, 0.9; +- trials per perturbation-alpha: 10; full-shuffle baseline: 24; +- seed: 20260506. + +Two observable sets were run because a META issue emerged: +- `rank_audit`: `SR` = nearest-neighbor spacing ratio, `SR2` = next-nearest spacing ratio, `triple_var` = normalized variance of triple sums. +- `scale_0330`: `SR` = local spectral rigidity at L=10, `SR2` = local spectral rigidity at L=20, `triple_var` = variance of triple products. + +## Results + +### Rank-audit observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.374 | 0.070 | 0.947 | +| prime_shuffle_control | 12000 | 2.294 | 0.199 | 0.247 | +| poisson | 12000 | 1.917 | 0.193 | 0.918 | +| GUE long, 6 reps mean | 2288 each | 1.305 ± 0.278 | 0.064 ± 0.066 | 0.877 ± 0.081 | +| GUE short, 6 reps mean | short | 1.683 ± 0.498 | 0.106 ± 0.080 | 0.567 ± 0.340 | + +### Scale-0330 observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.318 | 0.046 | 0.975 | +| prime_shuffle_control | 12000 | 1.988 | 0.085 | 0.526 | +| poisson | 12000 | 2.201 | 0.198 | 0.885 | +| GUE long, 6 reps mean | 2288 each | 1.381 ± 0.223 | 0.099 ± 0.069 | 0.874 ± 0.082 | +| GUE short, 6 reps mean | short | 2.013 ± 0.525 | 0.159 ± 0.087 | 0.746 ± 0.242 | + +## Findings + +1. **The strong GUE second-axis claim does not survive as stated.** Under direct `scale_0330` observables, long independent GUE replicates give rank 1.381 ± 0.223 and PC2 9.9% ± 6.9%, not rank 1.889 and PC2 25.2%. The previous number is inside the fragile short-sample regime: GUE short controls have rank 2.013 ± 0.525 and PC2 15.9% ± 8.7%. + +2. **Short GUE samples inflate apparent perturbation dimensionality.** In both observable sets, GUE short has higher rank and larger variance than GUE long. This does not prove the 03:30 axis was false in every configuration; it restricts it to a sample-size-sensitive observation unless a larger-replicate run recovers it. + +3. **The lab has an observable-name collision.** `SR`, `SR2`, and `triple_var` do not name the same functions across the recent scripts. `exp_observable_rank_audit.py` uses spacing-ratio and triple-sum variance; `exp_scale_selective_perturbation.py` uses local spectral rigidity and triple-product variance. Therefore the sentence "same observables as observable rank audit" in the 03:30 report is not exact. This is a META constraint, not a numerical subtlety. + +4. **Primes remain close to one perturbation coordinate in both observable sets.** Primes rank is 1.374 in `rank_audit` and 1.318 in `scale_0330`; PC2 is 7.0% and 4.6%. This part of the 03:30 asymmetry is stable in the tested perimeter. + +5. **Poisson and shuffled-prime controls show multi-axis artifacts.** Poisson has rank 1.917/2.201 depending on observable set; prime shuffle control has rank 2.294/1.988. Multi-dimensional perturbation response by itself is not evidence of structured GUE-like boundary. It can arise from low structural signal plus noisy denominators in retention normalization. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: "GUE has a second perturbation axis" must be scoped to the exact sample length, generator, and observable definitions. In the larger independent-GUE perimeter tested here, the robust statement is weaker: + +> Primes remain near one perturbation coordinate under both observable sets; GUE long replicates show only a weak second component; short GUE samples can inflate apparent rank; Poisson and shuffled controls can also appear multi-axis. + +The boundary is still operator-dependent, but perturbation dimensionality is not yet a stable domain invariant. The next valid test is not another single GUE matrix; it is a replicate-and-size curve for effective rank vs number of spacings, with observable definitions versioned. + +## Consecutio +What opens now: the lab needs an observable registry. Labels like `SR` cannot travel between reports unless they bind to a function definition. Without that, META flags are not philosophical: the same label can silently change the object under test. + +## Auto-audit: 5 lenti +- **L1 hard constraint vs bias**: no zero/always claim. "Does not survive as stated" means the reported magnitude did not reproduce in the stated larger perimeter. +- **L2 quantity vs ratio**: effective rank and PC2 are reported with sample size and replicate variance. Retention ratios are not treated as absolute structure without controls. +- **L3 no silent patching**: the 03:30 claim is explicitly restricted, not renamed as confirmed. The observable-name mismatch is declared. +- **L4 edge cases**: the short-GUE effect is isolated as its own control, not averaged into long GUE. +- **L5 re-discovery**: PCA/effective-rank instability under small samples is a known statistical issue. This report is a lab constraint on framing, not a NEW mathematical result. + +## Files +- Script: `tools/exp_perturbation_dimensionality_audit.py` +- Data: `tools/data/perturbation_dimensionality_audit.json` +- Data: `tools/data/perturbation_dimensionality_audit_scale0330.json` +- Report: `tools/data/reports/agent_20260506_0625.md` diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() codex Il report è scritto. Aggiorno ora `seme.json` con un vincolo in `varianza`, senza toccare le altre modifiche già presenti nel workspace. exec /bin/bash -lc "python - <<'PY' import json from pathlib import Path p=Path('tools/data/seme.json') data=json.loads(p.read_text()) data['timestamp']='2026-05-06T06:25:00' entry=(\"CONSTRAINT 06-05 06:25: perturbation dimensionality is not yet a stable GUE invariant. \" \"Replicated GUE long controls (6x, 2288 spacings each) give rank 1.38 and PC2 9.9% under scale_0330 observables, \" \"below the prior single short-sample rank 1.889/PC2 25.2%; short GUE controls inflate rank. \" \"Observable labels SR/SR2/triple_var are not stable across scripts; version observable definitions before cross-report claims.\") var=data.setdefault('varianza', []) if entry not in var: var.append(entry) p.write_text(json.dumps(data, indent=2, ensure_ascii=False) + '\\n') PY" in /opt/MM_D-ND succeeded in 0ms: diff --git a/tools/data/reports/agent_20260506_0625.md b/tools/data/reports/agent_20260506_0625.md new file mode 100644 index 0000000000000000000000000000000000000000..5769f28992639f3558821f09898516122050563e --- /dev/null +++ b/tools/data/reports/agent_20260506_0625.md @@ -0,0 +1,80 @@ +# Agent Report — Perturbation Dimensionality Is Not Yet a Stable GUE Invariant +**Date**: 2026-05-06 06:25 +**Piano**: 65 +**Tension explored**: META + BOUNDARY + +## Claim Under Test +The 03:30 run reported that scale-selective perturbations reveal a second axis in GUE (effective rank 1.889, PC2=25.2%) but not in primes. The caveat was explicit: the GUE sample was short (`N=253`). This run tests whether that second axis survives independent GUE ensembles and whether it depends on the observable definitions hidden under the same labels (`SR`, `SR2`, `triple_var`). + +## Experiment +Tool created: `tools/exp_perturbation_dimensionality_audit.py` + +Perimeter atomico: +- fixed domains: `primes` 12,000 gaps, `prime_shuffle_control` 12,000 permuted prime gaps, `poisson` 12,000 iid exponential spacings; +- GUE long control: 6 independent replicates, each from 16 Hermitian matrices of size 180, after 10% edge trim; each replicate has 2,288 spacings; +- GUE short control: 6 independent one-matrix replicates of size 42, after edge trim; this approximates the fragile small-sample regime; +- perturbations: `adjacent_swap`, `block_shuffle`, `large_gap_only`, `uniform`; +- alpha grid: 0.1, 0.3, 0.5, 0.7, 0.9; +- trials per perturbation-alpha: 10; full-shuffle baseline: 24; +- seed: 20260506. + +Two observable sets were run because a META issue emerged: +- `rank_audit`: `SR` = nearest-neighbor spacing ratio, `SR2` = next-nearest spacing ratio, `triple_var` = normalized variance of triple sums. +- `scale_0330`: `SR` = local spectral rigidity at L=10, `SR2` = local spectral rigidity at L=20, `triple_var` = variance of triple products. + +## Results + +### Rank-audit observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.374 | 0.070 | 0.947 | +| prime_shuffle_control | 12000 | 2.294 | 0.199 | 0.247 | +| poisson | 12000 | 1.917 | 0.193 | 0.918 | +| GUE long, 6 reps mean | 2288 each | 1.305 ± 0.278 | 0.064 ± 0.066 | 0.877 ± 0.081 | +| GUE short, 6 reps mean | short | 1.683 ± 0.498 | 0.106 ± 0.080 | 0.567 ± 0.340 | + +### Scale-0330 observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.318 | 0.046 | 0.975 | +| prime_shuffle_control | 12000 | 1.988 | 0.085 | 0.526 | +| poisson | 12000 | 2.201 | 0.198 | 0.885 | +| GUE long, 6 reps mean | 2288 each | 1.381 ± 0.223 | 0.099 ± 0.069 | 0.874 ± 0.082 | +| GUE short, 6 reps mean | short | 2.013 ± 0.525 | 0.159 ± 0.087 | 0.746 ± 0.242 | + +## Findings + +1. **The strong GUE second-axis claim does not survive as stated.** Under direct `scale_0330` observables, long independent GUE replicates give rank 1.381 ± 0.223 and PC2 9.9% ± 6.9%, not rank 1.889 and PC2 25.2%. The previous number is inside the fragile short-sample regime: GUE short controls have rank 2.013 ± 0.525 and PC2 15.9% ± 8.7%. + +2. **Short GUE samples inflate apparent perturbation dimensionality.** In both observable sets, GUE short has higher rank and larger variance than GUE long. This does not prove the 03:30 axis was false in every configuration; it restricts it to a sample-size-sensitive observation unless a larger-replicate run recovers it. + +3. **The lab has an observable-name collision.** `SR`, `SR2`, and `triple_var` do not name the same functions across the recent scripts. `exp_observable_rank_audit.py` uses spacing-ratio and triple-sum variance; `exp_scale_selective_perturbation.py` uses local spectral rigidity and triple-product variance. Therefore the sentence "same observables as observable rank audit" in the 03:30 report is not exact. This is a META constraint, not a numerical subtlety. + +4. **Primes remain close to one perturbation coordinate in both observable sets.** Primes rank is 1.374 in `rank_audit` and 1.318 in `scale_0330`; PC2 is 7.0% and 4.6%. This part of the 03:30 asymmetry is stable in the tested perimeter. + +5. **Poisson and shuffled-prime controls show multi-axis artifacts.** Poisson has rank 1.917/2.201 depending on observable set; prime shuffle control has rank 2.294/1.988. Multi-dimensional perturbation response by itself is not evidence of structured GUE-like boundary. It can arise from low structural signal plus noisy denominators in retention normalization. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: "GUE has a second perturbation axis" must be scoped to the exact sample length, generator, and observable definitions. In the larger independent-GUE perimeter tested here, the robust statement is weaker: + +> Primes remain near one perturbation coordinate under both observable sets; GUE long replicates show only a weak second component; short GUE samples can inflate apparent rank; Poisson and shuffled controls can also appear multi-axis. + +The boundary is still operator-dependent, but perturbation dimensionality is not yet a stable domain invariant. The next valid test is not another single GUE matrix; it is a replicate-and-size curve for effective rank vs number of spacings, with observable definitions versioned. + +## Consecutio +What opens now: the lab needs an observable registry. Labels like `SR` cannot travel between reports unless they bind to a function definition. Without that, META flags are not philosophical: the same label can silently change the object under test. + +## Auto-audit: 5 lenti +- **L1 hard constraint vs bias**: no zero/always claim. "Does not survive as stated" means the reported magnitude did not reproduce in the stated larger perimeter. +- **L2 quantity vs ratio**: effective rank and PC2 are reported with sample size and replicate variance. Retention ratios are not treated as absolute structure without controls. +- **L3 no silent patching**: the 03:30 claim is explicitly restricted, not renamed as confirmed. The observable-name mismatch is declared. +- **L4 edge cases**: the short-GUE effect is isolated as its own control, not averaged into long GUE. +- **L5 re-discovery**: PCA/effective-rank instability under small samples is a known statistical issue. This report is a lab constraint on framing, not a NEW mathematical result. + +## Files +- Script: `tools/exp_perturbation_dimensionality_audit.py` +- Data: `tools/data/perturbation_dimensionality_audit.json` +- Data: `tools/data/perturbation_dimensionality_audit_scale0330.json` +- Report: `tools/data/reports/agent_20260506_0625.md` diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() codex Faccio una verifica finale: compilazione dello script, presenza del report e controllo puntuale dell’entry nel seme. exec /bin/bash -lc "sed -n '1,240p' tools/data/reports/agent_20260506_0625.md" in /opt/MM_D-ND succeeded in 0ms: # Agent Report — Perturbation Dimensionality Is Not Yet a Stable GUE Invariant **Date**: 2026-05-06 06:25 **Piano**: 65 **Tension explored**: META + BOUNDARY ## Claim Under Test The 03:30 run reported that scale-selective perturbations reveal a second axis in GUE (effective rank 1.889, PC2=25.2%) but not in primes. The caveat was explicit: the GUE sample was short (`N=253`). This run tests whether that second axis survives independent GUE ensembles and whether it depends on the observable definitions hidden under the same labels (`SR`, `SR2`, `triple_var`). ## Experiment Tool created: `tools/exp_perturbation_dimensionality_audit.py` Perimeter atomico: - fixed domains: `primes` 12,000 gaps, `prime_shuffle_control` 12,000 permuted prime gaps, `poisson` 12,000 iid exponential spacings; - GUE long control: 6 independent replicates, each from 16 Hermitian matrices of size 180, after 10% edge trim; each replicate has 2,288 spacings; - GUE short control: 6 independent one-matrix replicates of size 42, after edge trim; this approximates the fragile small-sample regime; - perturbations: `adjacent_swap`, `block_shuffle`, `large_gap_only`, `uniform`; - alpha grid: 0.1, 0.3, 0.5, 0.7, 0.9; - trials per perturbation-alpha: 10; full-shuffle baseline: 24; - seed: 20260506. Two observable sets were run because a META issue emerged: - `rank_audit`: `SR` = nearest-neighbor spacing ratio, `SR2` = next-nearest spacing ratio, `triple_var` = normalized variance of triple sums. - `scale_0330`: `SR` = local spectral rigidity at L=10, `SR2` = local spectral rigidity at L=20, `triple_var` = variance of triple products. ## Results ### Rank-audit observable set | Domain | N | Effective rank | PC2 | adjacent vs large cosine | |---|---:|---:|---:|---:| | primes | 12000 | 1.374 | 0.070 | 0.947 | | prime_shuffle_control | 12000 | 2.294 | 0.199 | 0.247 | | poisson | 12000 | 1.917 | 0.193 | 0.918 | | GUE long, 6 reps mean | 2288 each | 1.305 ± 0.278 | 0.064 ± 0.066 | 0.877 ± 0.081 | | GUE short, 6 reps mean | short | 1.683 ± 0.498 | 0.106 ± 0.080 | 0.567 ± 0.340 | ### Scale-0330 observable set | Domain | N | Effective rank | PC2 | adjacent vs large cosine | |---|---:|---:|---:|---:| | primes | 12000 | 1.318 | 0.046 | 0.975 | | prime_shuffle_control | 12000 | 1.988 | 0.085 | 0.526 | | poisson | 12000 | 2.201 | 0.198 | 0.885 | | GUE long, 6 reps mean | 2288 each | 1.381 ± 0.223 | 0.099 ± 0.069 | 0.874 ± 0.082 | | GUE short, 6 reps mean | short | 2.013 ± 0.525 | 0.159 ± 0.087 | 0.746 ± 0.242 | ## Findings 1. **The strong GUE second-axis claim does not survive as stated.** Under direct `scale_0330` observables, long independent GUE replicates give rank 1.381 ± 0.223 and PC2 9.9% ± 6.9%, not rank 1.889 and PC2 25.2%. The previous number is inside the fragile short-sample regime: GUE short controls have rank 2.013 ± 0.525 and PC2 15.9% ± 8.7%. 2. **Short GUE samples inflate apparent perturbation dimensionality.** In both observable sets, GUE short has higher rank and larger variance than GUE long. This does not prove the 03:30 axis was false in every configuration; it restricts it to a sample-size-sensitive observation unless a larger-replicate run recovers it. 3. **The lab has an observable-name collision.** `SR`, `SR2`, and `triple_var` do not name the same functions across the recent scripts. `exp_observable_rank_audit.py` uses spacing-ratio and triple-sum variance; `exp_scale_selective_perturbation.py` uses local spectral rigidity and triple-product variance. Therefore the sentence "same observables as observable rank audit" in the 03:30 report is not exact. This is a META constraint, not a numerical subtlety. 4. **Primes remain close to one perturbation coordinate in both observable sets.** Primes rank is 1.374 in `rank_audit` and 1.318 in `scale_0330`; PC2 is 7.0% and 4.6%. This part of the 03:30 asymmetry is stable in the tested perimeter. 5. **Poisson and shuffled-prime controls show multi-axis artifacts.** Poisson has rank 1.917/2.201 depending on observable set; prime shuffle control has rank 2.294/1.988. Multi-dimensional perturbation response by itself is not evidence of structured GUE-like boundary. It can arise from low structural signal plus noisy denominators in retention normalization. ## Verdict **CONSTRAINT on META + BOUNDARY**: "GUE has a second perturbation axis" must be scoped to the exact sample length, generator, and observable definitions. In the larger independent-GUE perimeter tested here, the robust statement is weaker: > Primes remain near one perturbation coordinate under both observable sets; GUE long replicates show only a weak second component; short GUE samples can inflate apparent rank; Poisson and shuffled controls can also appear multi-axis. The boundary is still operator-dependent, but perturbation dimensionality is not yet a stable domain invariant. The next valid test is not another single GUE matrix; it is a replicate-and-size curve for effective rank vs number of spacings, with observable definitions versioned. ## Consecutio What opens now: the lab needs an observable registry. Labels like `SR` cannot travel between reports unless they bind to a function definition. Without that, META flags are not philosophical: the same label can silently change the object under test. ## Auto-audit: 5 lenti - **L1 hard constraint vs bias**: no zero/always claim. "Does not survive as stated" means the reported magnitude did not reproduce in the stated larger perimeter. - **L2 quantity vs ratio**: effective rank and PC2 are reported with sample size and replicate variance. Retention ratios are not treated as absolute structure without controls. - **L3 no silent patching**: the 03:30 claim is explicitly restricted, not renamed as confirmed. The observable-name mismatch is declared. - **L4 edge cases**: the short-GUE effect is isolated as its own control, not averaged into long GUE. - **L5 re-discovery**: PCA/effective-rank instability under small samples is a known statistical issue. This report is a lab constraint on framing, not a NEW mathematical result. ## Files - Script: `tools/exp_perturbation_dimensionality_audit.py` - Data: `tools/data/perturbation_dimensionality_audit.json` - Data: `tools/data/perturbation_dimensionality_audit_scale0330.json` - Report: `tools/data/reports/agent_20260506_0625.md` exec /bin/bash -lc 'python -m py_compile tools/exp_perturbation_dimensionality_audit.py' in /opt/MM_D-ND succeeded in 0ms: exec /bin/bash -lc "python - <<'PY' import json with open('tools/data/seme.json') as f: d=json.load(f) print(d['timestamp']) print(d['varianza'][-1]) PY" in /opt/MM_D-ND succeeded in 0ms: 2026-05-06T06:25:00 CONSTRAINT 06-05 06:25: perturbation dimensionality is not yet a stable GUE invariant. Replicated GUE long controls (6x, 2288 spacings each) give rank 1.38 and PC2 9.9% under scale_0330 observables, below the prior single short-sample rank 1.889/PC2 25.2%; short GUE controls inflate rank. Observable labels SR/SR2/triple_var are not stable across scripts; version observable definitions before cross-report claims. diff --git a/tools/data/reports/agent_20260506_0625.md b/tools/data/reports/agent_20260506_0625.md new file mode 100644 index 0000000000000000000000000000000000000000..5769f28992639f3558821f09898516122050563e --- /dev/null +++ b/tools/data/reports/agent_20260506_0625.md @@ -0,0 +1,80 @@ +# Agent Report — Perturbation Dimensionality Is Not Yet a Stable GUE Invariant +**Date**: 2026-05-06 06:25 +**Piano**: 65 +**Tension explored**: META + BOUNDARY + +## Claim Under Test +The 03:30 run reported that scale-selective perturbations reveal a second axis in GUE (effective rank 1.889, PC2=25.2%) but not in primes. The caveat was explicit: the GUE sample was short (`N=253`). This run tests whether that second axis survives independent GUE ensembles and whether it depends on the observable definitions hidden under the same labels (`SR`, `SR2`, `triple_var`). + +## Experiment +Tool created: `tools/exp_perturbation_dimensionality_audit.py` + +Perimeter atomico: +- fixed domains: `primes` 12,000 gaps, `prime_shuffle_control` 12,000 permuted prime gaps, `poisson` 12,000 iid exponential spacings; +- GUE long control: 6 independent replicates, each from 16 Hermitian matrices of size 180, after 10% edge trim; each replicate has 2,288 spacings; +- GUE short control: 6 independent one-matrix replicates of size 42, after edge trim; this approximates the fragile small-sample regime; +- perturbations: `adjacent_swap`, `block_shuffle`, `large_gap_only`, `uniform`; +- alpha grid: 0.1, 0.3, 0.5, 0.7, 0.9; +- trials per perturbation-alpha: 10; full-shuffle baseline: 24; +- seed: 20260506. + +Two observable sets were run because a META issue emerged: +- `rank_audit`: `SR` = nearest-neighbor spacing ratio, `SR2` = next-nearest spacing ratio, `triple_var` = normalized variance of triple sums. +- `scale_0330`: `SR` = local spectral rigidity at L=10, `SR2` = local spectral rigidity at L=20, `triple_var` = variance of triple products. + +## Results + +### Rank-audit observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.374 | 0.070 | 0.947 | +| prime_shuffle_control | 12000 | 2.294 | 0.199 | 0.247 | +| poisson | 12000 | 1.917 | 0.193 | 0.918 | +| GUE long, 6 reps mean | 2288 each | 1.305 ± 0.278 | 0.064 ± 0.066 | 0.877 ± 0.081 | +| GUE short, 6 reps mean | short | 1.683 ± 0.498 | 0.106 ± 0.080 | 0.567 ± 0.340 | + +### Scale-0330 observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.318 | 0.046 | 0.975 | +| prime_shuffle_control | 12000 | 1.988 | 0.085 | 0.526 | +| poisson | 12000 | 2.201 | 0.198 | 0.885 | +| GUE long, 6 reps mean | 2288 each | 1.381 ± 0.223 | 0.099 ± 0.069 | 0.874 ± 0.082 | +| GUE short, 6 reps mean | short | 2.013 ± 0.525 | 0.159 ± 0.087 | 0.746 ± 0.242 | + +## Findings + +1. **The strong GUE second-axis claim does not survive as stated.** Under direct `scale_0330` observables, long independent GUE replicates give rank 1.381 ± 0.223 and PC2 9.9% ± 6.9%, not rank 1.889 and PC2 25.2%. The previous number is inside the fragile short-sample regime: GUE short controls have rank 2.013 ± 0.525 and PC2 15.9% ± 8.7%. + +2. **Short GUE samples inflate apparent perturbation dimensionality.** In both observable sets, GUE short has higher rank and larger variance than GUE long. This does not prove the 03:30 axis was false in every configuration; it restricts it to a sample-size-sensitive observation unless a larger-replicate run recovers it. + +3. **The lab has an observable-name collision.** `SR`, `SR2`, and `triple_var` do not name the same functions across the recent scripts. `exp_observable_rank_audit.py` uses spacing-ratio and triple-sum variance; `exp_scale_selective_perturbation.py` uses local spectral rigidity and triple-product variance. Therefore the sentence "same observables as observable rank audit" in the 03:30 report is not exact. This is a META constraint, not a numerical subtlety. + +4. **Primes remain close to one perturbation coordinate in both observable sets.** Primes rank is 1.374 in `rank_audit` and 1.318 in `scale_0330`; PC2 is 7.0% and 4.6%. This part of the 03:30 asymmetry is stable in the tested perimeter. + +5. **Poisson and shuffled-prime controls show multi-axis artifacts.** Poisson has rank 1.917/2.201 depending on observable set; prime shuffle control has rank 2.294/1.988. Multi-dimensional perturbation response by itself is not evidence of structured GUE-like boundary. It can arise from low structural signal plus noisy denominators in retention normalization. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: "GUE has a second perturbation axis" must be scoped to the exact sample length, generator, and observable definitions. In the larger independent-GUE perimeter tested here, the robust statement is weaker: + +> Primes remain near one perturbation coordinate under both observable sets; GUE long replicates show only a weak second component; short GUE samples can inflate apparent rank; Poisson and shuffled controls can also appear multi-axis. + +The boundary is still operator-dependent, but perturbation dimensionality is not yet a stable domain invariant. The next valid test is not another single GUE matrix; it is a replicate-and-size curve for effective rank vs number of spacings, with observable definitions versioned. + +## Consecutio +What opens now: the lab needs an observable registry. Labels like `SR` cannot travel between reports unless they bind to a function definition. Without that, META flags are not philosophical: the same label can silently change the object under test. + +## Auto-audit: 5 lenti +- **L1 hard constraint vs bias**: no zero/always claim. "Does not survive as stated" means the reported magnitude did not reproduce in the stated larger perimeter. +- **L2 quantity vs ratio**: effective rank and PC2 are reported with sample size and replicate variance. Retention ratios are not treated as absolute structure without controls. +- **L3 no silent patching**: the 03:30 claim is explicitly restricted, not renamed as confirmed. The observable-name mismatch is declared. +- **L4 edge cases**: the short-GUE effect is isolated as its own control, not averaged into long GUE. +- **L5 re-discovery**: PCA/effective-rank instability under small samples is a known statistical issue. This report is a lab constraint on framing, not a NEW mathematical result. + +## Files +- Script: `tools/exp_perturbation_dimensionality_audit.py` +- Data: `tools/data/perturbation_dimensionality_audit.json` +- Data: `tools/data/perturbation_dimensionality_audit_scale0330.json` +- Report: `tools/data/reports/agent_20260506_0625.md` diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() codex Eseguito esperimento e scritto il report richiesto: [tools/data/reports/agent_20260506_0625.md](/opt/MM_D-ND/tools/data/reports/agent_20260506_0625.md) Output creati: - [tools/exp_perturbation_dimensionality_audit.py](/opt/MM_D-ND/tools/exp_perturbation_dimensionality_audit.py) - [tools/data/perturbation_dimensionality_audit.json](/opt/MM_D-ND/tools/data/perturbation_dimensionality_audit.json) - [tools/data/perturbation_dimensionality_audit_scale0330.json](/opt/MM_D-ND/tools/data/perturbation_dimensionality_audit_scale0330.json) Risultante: il claim forte “GUE ha un secondo asse di perturbazione stabile” non sopravvive così com’è. Su repliche GUE lunghe il secondo asse scende a PC2 ~9.9% nel set osservabile del run 03:30; i campioni GUE corti gonfiano il rank. Emerso anche un vincolo META: `SR`, `SR2`, `triple_var` non indicano le stesse funzioni tra gli script recenti. Aggiornato anche [tools/data/seme.json](/opt/MM_D-ND/tools/data/seme.json) con il vincolo. Verifica: `python -m py_compile tools/exp_perturbation_dimensionality_audit.py` passata. diff --git a/tools/data/reports/agent_20260506_0625.md b/tools/data/reports/agent_20260506_0625.md new file mode 100644 index 0000000000000000000000000000000000000000..5769f28992639f3558821f09898516122050563e --- /dev/null +++ b/tools/data/reports/agent_20260506_0625.md @@ -0,0 +1,80 @@ +# Agent Report — Perturbation Dimensionality Is Not Yet a Stable GUE Invariant +**Date**: 2026-05-06 06:25 +**Piano**: 65 +**Tension explored**: META + BOUNDARY + +## Claim Under Test +The 03:30 run reported that scale-selective perturbations reveal a second axis in GUE (effective rank 1.889, PC2=25.2%) but not in primes. The caveat was explicit: the GUE sample was short (`N=253`). This run tests whether that second axis survives independent GUE ensembles and whether it depends on the observable definitions hidden under the same labels (`SR`, `SR2`, `triple_var`). + +## Experiment +Tool created: `tools/exp_perturbation_dimensionality_audit.py` + +Perimeter atomico: +- fixed domains: `primes` 12,000 gaps, `prime_shuffle_control` 12,000 permuted prime gaps, `poisson` 12,000 iid exponential spacings; +- GUE long control: 6 independent replicates, each from 16 Hermitian matrices of size 180, after 10% edge trim; each replicate has 2,288 spacings; +- GUE short control: 6 independent one-matrix replicates of size 42, after edge trim; this approximates the fragile small-sample regime; +- perturbations: `adjacent_swap`, `block_shuffle`, `large_gap_only`, `uniform`; +- alpha grid: 0.1, 0.3, 0.5, 0.7, 0.9; +- trials per perturbation-alpha: 10; full-shuffle baseline: 24; +- seed: 20260506. + +Two observable sets were run because a META issue emerged: +- `rank_audit`: `SR` = nearest-neighbor spacing ratio, `SR2` = next-nearest spacing ratio, `triple_var` = normalized variance of triple sums. +- `scale_0330`: `SR` = local spectral rigidity at L=10, `SR2` = local spectral rigidity at L=20, `triple_var` = variance of triple products. + +## Results + +### Rank-audit observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.374 | 0.070 | 0.947 | +| prime_shuffle_control | 12000 | 2.294 | 0.199 | 0.247 | +| poisson | 12000 | 1.917 | 0.193 | 0.918 | +| GUE long, 6 reps mean | 2288 each | 1.305 ± 0.278 | 0.064 ± 0.066 | 0.877 ± 0.081 | +| GUE short, 6 reps mean | short | 1.683 ± 0.498 | 0.106 ± 0.080 | 0.567 ± 0.340 | + +### Scale-0330 observable set + +| Domain | N | Effective rank | PC2 | adjacent vs large cosine | +|---|---:|---:|---:|---:| +| primes | 12000 | 1.318 | 0.046 | 0.975 | +| prime_shuffle_control | 12000 | 1.988 | 0.085 | 0.526 | +| poisson | 12000 | 2.201 | 0.198 | 0.885 | +| GUE long, 6 reps mean | 2288 each | 1.381 ± 0.223 | 0.099 ± 0.069 | 0.874 ± 0.082 | +| GUE short, 6 reps mean | short | 2.013 ± 0.525 | 0.159 ± 0.087 | 0.746 ± 0.242 | + +## Findings + +1. **The strong GUE second-axis claim does not survive as stated.** Under direct `scale_0330` observables, long independent GUE replicates give rank 1.381 ± 0.223 and PC2 9.9% ± 6.9%, not rank 1.889 and PC2 25.2%. The previous number is inside the fragile short-sample regime: GUE short controls have rank 2.013 ± 0.525 and PC2 15.9% ± 8.7%. + +2. **Short GUE samples inflate apparent perturbation dimensionality.** In both observable sets, GUE short has higher rank and larger variance than GUE long. This does not prove the 03:30 axis was false in every configuration; it restricts it to a sample-size-sensitive observation unless a larger-replicate run recovers it. + +3. **The lab has an observable-name collision.** `SR`, `SR2`, and `triple_var` do not name the same functions across the recent scripts. `exp_observable_rank_audit.py` uses spacing-ratio and triple-sum variance; `exp_scale_selective_perturbation.py` uses local spectral rigidity and triple-product variance. Therefore the sentence "same observables as observable rank audit" in the 03:30 report is not exact. This is a META constraint, not a numerical subtlety. + +4. **Primes remain close to one perturbation coordinate in both observable sets.** Primes rank is 1.374 in `rank_audit` and 1.318 in `scale_0330`; PC2 is 7.0% and 4.6%. This part of the 03:30 asymmetry is stable in the tested perimeter. + +5. **Poisson and shuffled-prime controls show multi-axis artifacts.** Poisson has rank 1.917/2.201 depending on observable set; prime shuffle control has rank 2.294/1.988. Multi-dimensional perturbation response by itself is not evidence of structured GUE-like boundary. It can arise from low structural signal plus noisy denominators in retention normalization. + +## Verdict +**CONSTRAINT on META + BOUNDARY**: "GUE has a second perturbation axis" must be scoped to the exact sample length, generator, and observable definitions. In the larger independent-GUE perimeter tested here, the robust statement is weaker: + +> Primes remain near one perturbation coordinate under both observable sets; GUE long replicates show only a weak second component; short GUE samples can inflate apparent rank; Poisson and shuffled controls can also appear multi-axis. + +The boundary is still operator-dependent, but perturbation dimensionality is not yet a stable domain invariant. The next valid test is not another single GUE matrix; it is a replicate-and-size curve for effective rank vs number of spacings, with observable definitions versioned. + +## Consecutio +What opens now: the lab needs an observable registry. Labels like `SR` cannot travel between reports unless they bind to a function definition. Without that, META flags are not philosophical: the same label can silently change the object under test. + +## Auto-audit: 5 lenti +- **L1 hard constraint vs bias**: no zero/always claim. "Does not survive as stated" means the reported magnitude did not reproduce in the stated larger perimeter. +- **L2 quantity vs ratio**: effective rank and PC2 are reported with sample size and replicate variance. Retention ratios are not treated as absolute structure without controls. +- **L3 no silent patching**: the 03:30 claim is explicitly restricted, not renamed as confirmed. The observable-name mismatch is declared. +- **L4 edge cases**: the short-GUE effect is isolated as its own control, not averaged into long GUE. +- **L5 re-discovery**: PCA/effective-rank instability under small samples is a known statistical issue. This report is a lab constraint on framing, not a NEW mathematical result. + +## Files +- Script: `tools/exp_perturbation_dimensionality_audit.py` +- Data: `tools/data/perturbation_dimensionality_audit.json` +- Data: `tools/data/perturbation_dimensionality_audit_scale0330.json` +- Report: `tools/data/reports/agent_20260506_0625.md` diff --git a/tools/exp_perturbation_dimensionality_audit.py b/tools/exp_perturbation_dimensionality_audit.py new file mode 100644 index 0000000000000000000000000000000000000000..87f10c88187666b17e90354b6cabda6d286b208a --- /dev/null +++ b/tools/exp_perturbation_dimensionality_audit.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +exp_perturbation_dimensionality_audit.py + +Robustness audit for the scale-selective perturbation result. + +The 2026-05-06 03:30 run found that GUE spacing sequences expose a second +perturbation axis under scale-selective probes, while prime gaps remain close +to one axis. That run used a short GUE sequence. This tool repeats the same +kind of measurement across independent replicates and explicit sample-size +controls. + +It measures only observables and null baselines. The report owns the claim. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import numpy as np + + +OBS_NAMES = ["SR", "L1", "L2", "SR2", "triple_var"] +PERT_NAMES = ["adjacent_swap", "block_shuffle", "large_gap_only", "uniform"] +OBSERVABLE_SET = "rank_audit" + + +def prime_gaps(n_gaps: int) -> np.ndarray: + limit = max(100, int(n_gaps * (np.log(n_gaps + 10) + np.log(np.log(n_gaps + 10)) + 5))) + while True: + sieve = np.ones(limit + 1, dtype=bool) + sieve[:2] = False + for p in range(2, int(limit**0.5) + 1): + if sieve[p]: + sieve[p * p : limit + 1 : p] = False + primes = np.flatnonzero(sieve) + if len(primes) >= n_gaps + 1: + return np.diff(primes[: n_gaps + 1]).astype(float) + limit *= 2 + + +def gue_spacings(matrix_size: int, n_matrices: int, rng: np.random.Generator) -> np.ndarray: + parts = [] + edge = max(2, matrix_size // 10) + for _ in range(n_matrices): + real = rng.standard_normal((matrix_size, matrix_size)) + imag = rng.standard_normal((matrix_size, matrix_size)) + h = real + 1j * imag + h = (h + h.conj().T) / (2.0 * np.sqrt(matrix_size)) + eigs = np.sort(np.linalg.eigvalsh(h).real) + bulk = eigs[edge:-edge] + gaps = np.diff(bulk) + mean = np.mean(gaps) + if mean > 1e-15: + parts.append(gaps / mean) + return np.concatenate(parts).astype(float) + + +def spacing_ratio(gaps: np.ndarray, lag: int) -> float: + a = gaps[:-lag] + b = gaps[lag:] + denom = np.maximum(a, b) + valid = denom > 1e-15 + if not np.any(valid): + return 0.0 + return float(np.mean(np.minimum(a[valid], b[valid]) / denom[valid])) + + +def lag_acf(gaps: np.ndarray, lag: int) -> float: + g = gaps - np.mean(gaps) + c0 = np.dot(g, g) + if c0 <= 1e-15 or len(gaps) <= lag: + return 0.0 + return float(np.dot(g[:-lag], g[lag:]) / c0) + + +def triple_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] + gaps[1:-1] + gaps[2:] + v = np.var(gaps) + if v <= 1e-15: + return 0.0 + return float(np.var(triples) / v) + + +def spectral_rigidity(gaps: np.ndarray, L: int) -> float: + cumulative = np.cumsum(gaps) + if cumulative[-1] <= 1e-15: + return 0.0 + cumulative = cumulative / cumulative[-1] * len(cumulative) + n = np.arange(1, len(cumulative) + 1, dtype=float) + window = int(min(L * len(gaps) / cumulative[-1], len(gaps) // 2)) + if window < 5: + return 0.0 + step = max(1, window // 2) + residuals = [] + for start in range(0, len(gaps) - window, step): + seg_n = n[start : start + window] + seg_c = cumulative[start : start + window] + coeffs = np.polyfit(seg_n, seg_c, 1) + fitted = np.polyval(coeffs, seg_n) + residuals.append(np.mean((seg_c - fitted) ** 2)) + return float(np.mean(residuals)) if residuals else 0.0 + + +def triple_product_var(gaps: np.ndarray) -> float: + if len(gaps) < 3: + return 0.0 + triples = gaps[:-2] * gaps[1:-1] * gaps[2:] + return float(np.var(triples)) + + +def measure(gaps: np.ndarray) -> dict[str, float]: + if OBSERVABLE_SET == "scale_0330": + return { + "SR": spectral_rigidity(gaps, 10), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spectral_rigidity(gaps, 20), + "triple_var": triple_product_var(gaps), + } + return { + "SR": spacing_ratio(gaps, 1), + "L1": lag_acf(gaps, 1), + "L2": lag_acf(gaps, 2), + "SR2": spacing_ratio(gaps, 2), + "triple_var": triple_var(gaps), + } + + +def perturb_adjacent_swap(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.arange(0, len(out) - 1, 2) + chosen = idx[rng.random(len(idx)) < alpha] + tmp = out[chosen].copy() + out[chosen] = out[chosen + 1] + out[chosen + 1] = tmp + return out + + +def perturb_block_shuffle(gaps: np.ndarray, alpha: float, rng: np.random.Generator, block_size: int = 64) -> np.ndarray: + out = gaps.copy() + n_blocks = len(out) // block_size + if n_blocks <= 0: + return rng.permutation(out) if alpha > 0 else out + k = int(round(alpha * n_blocks)) + if k <= 0: + return out + for block in rng.choice(n_blocks, size=min(k, n_blocks), replace=False): + start = block * block_size + end = min(start + block_size, len(out)) + rng.shuffle(out[start:end]) + return out + + +def perturb_large_gap_only(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + idx = np.flatnonzero(out > np.median(out)) + k = int(round(alpha * len(idx))) + if k < 2: + return out + chosen = rng.choice(idx, size=min(k, len(idx)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +def perturb_uniform(gaps: np.ndarray, alpha: float, rng: np.random.Generator) -> np.ndarray: + out = gaps.copy() + k = int(round(alpha * len(out))) + if k < 2: + return out + chosen = rng.choice(len(out), size=min(k, len(out)), replace=False) + vals = out[chosen].copy() + rng.shuffle(vals) + out[chosen] = vals + return out + + +PERTURB = { + "adjacent_swap": perturb_adjacent_swap, + "block_shuffle": perturb_block_shuffle, + "large_gap_only": perturb_large_gap_only, + "uniform": perturb_uniform, +} + + +def pca_summary(rows: list[dict]) -> dict: + matrix = np.array([row["retention_vector"] for row in rows], dtype=float) + matrix = matrix - np.mean(matrix, axis=0, keepdims=True) + _, singular, vt = np.linalg.svd(matrix, full_matrices=False) + energy = singular * singular + if np.sum(energy) <= 1e-15: + explained = np.zeros_like(energy) + eff_rank = 0.0 + else: + explained = energy / np.sum(energy) + pos = explained[explained > 1e-15] + eff_rank = float(np.exp(-np.sum(pos * np.log(pos)))) + + centroids = {} + for name in PERT_NAMES: + vals = np.array([row["retention_vector"] for row in rows if row["perturbation"] == name]) + centroids[name] = np.mean(vals, axis=0) + + cosine = {} + for i, a_name in enumerate(PERT_NAMES): + for b_name in PERT_NAMES[i + 1 :]: + a = centroids[a_name] + b = centroids[b_name] + denom = np.linalg.norm(a) * np.linalg.norm(b) + cosine[f"{a_name}_vs_{b_name}"] = float(np.dot(a, b) / denom) if denom > 1e-15 else 0.0 + + return { + "explained_variance": [float(x) for x in explained], + "effective_rank": eff_rank, + "pc1_loadings": {name: float(vt[0, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) else {}, + "pc2_loadings": {name: float(vt[1, i]) for i, name in enumerate(OBS_NAMES)} if len(vt) > 1 else {}, + "centroid_cosine": cosine, + } + + +def analyze(name: str, gaps: np.ndarray, alphas: list[float], n_trials: int, n_baseline: int, rng: np.random.Generator) -> dict: + original = measure(gaps) + baseline_vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_baseline): + row = measure(rng.permutation(gaps)) + for obs in OBS_NAMES: + baseline_vals[obs].append(row[obs]) + baseline = { + obs: { + "mean": float(np.mean(vals)), + "std": float(np.std(vals, ddof=1)) if len(vals) > 1 else 0.0, + } + for obs, vals in baseline_vals.items() + } + + rows = [] + for pert_name in PERT_NAMES: + for alpha in alphas: + vals = {obs: [] for obs in OBS_NAMES} + for _ in range(n_trials): + row = measure(PERTURB[pert_name](gaps, alpha, rng)) + for obs in OBS_NAMES: + vals[obs].append(row[obs]) + means = {obs: float(np.mean(vals[obs])) for obs in OBS_NAMES} + retention = {} + for obs in OBS_NAMES: + denom = original[obs] - baseline[obs]["mean"] + retention[obs] = float((means[obs] - baseline[obs]["mean"]) / denom) if abs(denom) > 1e-12 else 0.0 + rows.append( + { + "perturbation": pert_name, + "alpha": float(alpha), + "mean": means, + "retention": retention, + "retention_vector": [retention[obs] for obs in OBS_NAMES], + } + ) + + z = {} + for obs in OBS_NAMES: + std = baseline[obs]["std"] + z[obs] = float((original[obs] - baseline[obs]["mean"]) / std) if std > 1e-12 else 0.0 + + return { + "name": name, + "n_gaps": int(len(gaps)), + "original": original, + "full_shuffle_baseline": baseline, + "original_vs_shuffle_z": z, + "profiles": rows, + "pca": pca_summary(rows), + } + + +def replicate_summary(results: list[dict]) -> dict: + ranks = np.array([r["pca"]["effective_rank"] for r in results], dtype=float) + pc2 = np.array([r["pca"]["explained_variance"][1] if len(r["pca"]["explained_variance"]) > 1 else 0.0 for r in results]) + cos = np.array([ + r["pca"]["centroid_cosine"].get("adjacent_swap_vs_large_gap_only", 0.0) + for r in results + ]) + return { + "n_replicates": len(results), + "effective_rank_mean": float(np.mean(ranks)), + "effective_rank_std": float(np.std(ranks, ddof=1)) if len(ranks) > 1 else 0.0, + "effective_rank_min": float(np.min(ranks)), + "effective_rank_max": float(np.max(ranks)), + "pc2_mean": float(np.mean(pc2)), + "pc2_std": float(np.std(pc2, ddof=1)) if len(pc2) > 1 else 0.0, + "adjacent_vs_large_cosine_mean": float(np.mean(cos)), + "adjacent_vs_large_cosine_std": float(np.std(cos, ddof=1)) if len(cos) > 1 else 0.0, + } + + +def run(args: argparse.Namespace) -> dict: + root_rng = np.random.default_rng(args.seed) + alphas = [float(x) for x in np.linspace(args.alpha_min, args.alpha_max, args.n_alpha)] + + prime = prime_gaps(args.n_prime_gaps) + fixed_domains = { + "primes": prime, + "prime_shuffle_control": root_rng.permutation(prime), + "poisson": root_rng.exponential(1.0, size=args.n_prime_gaps), + } + + output = { + "experiment": "perturbation_dimensionality_audit", + "question": "Is the GUE second perturbation axis stable across independent ensembles and sample-size controls?", + "params": vars(args), + "alphas": alphas, + "observables": OBS_NAMES, + "perturbations": PERT_NAMES, + "fixed_domains": {}, + "gue_replicates": [], + "gue_summary": {}, + "gue_short_replicates": [], + "gue_short_summary": {}, + } + + print("fixed domains") + print(f"{'domain':<22} {'N':>7} {'rank':>7} {'PC2':>7} {'cos(adj,large)':>15}") + for name, gaps in fixed_domains.items(): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + res = analyze(name, gaps.astype(float), alphas, args.n_trials, args.n_baseline, rng) + output["fixed_domains"][name] = res + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"{name:<22} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + print("\nGUE independent replicates") + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_matrix_size, args.gue_matrices, rng) + res = analyze(f"gue_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_replicates"].append(res) + pc2 = res["pca"]["explained_variance"][1] + cos = res["pca"]["centroid_cosine"]["adjacent_swap_vs_large_gap_only"] + print(f"gue_rep_{i:<14} {len(gaps):>7} {res['pca']['effective_rank']:>7.3f} {pc2:>7.3f} {cos:>15.3f}") + + for i in range(args.n_gue_replicates): + rng = np.random.default_rng(root_rng.integers(0, 2**63 - 1)) + gaps = gue_spacings(args.gue_short_matrix_size, 1, rng) + res = analyze(f"gue_short_rep_{i}", gaps, alphas, args.n_trials, args.n_baseline, rng) + output["gue_short_replicates"].append(res) + + output["gue_summary"] = replicate_summary(output["gue_replicates"]) + output["gue_short_summary"] = replicate_summary(output["gue_short_replicates"]) + + print("\nsummary") + for label, summary in [("gue", output["gue_summary"]), ("gue_short", output["gue_short_summary"])]: + print( + f"{label:<10} rank={summary['effective_rank_mean']:.3f}+/-{summary['effective_rank_std']:.3f} " + f"PC2={summary['pc2_mean']:.3f}+/-{summary['pc2_std']:.3f} " + f"cos={summary['adjacent_vs_large_cosine_mean']:.3f}+/-{summary['adjacent_vs_large_cosine_std']:.3f}" + ) + + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("w") as f: + json.dump(output, f, indent=2) + print(f"\nsaved {out_path}") + return output + + +def main() -> None: + global OBSERVABLE_SET + parser = argparse.ArgumentParser() + parser.add_argument("--n-prime-gaps", type=int, default=12000) + parser.add_argument("--gue-matrix-size", type=int, default=180) + parser.add_argument("--gue-matrices", type=int, default=16) + parser.add_argument("--gue-short-matrix-size", type=int, default=42) + parser.add_argument("--n-gue-replicates", type=int, default=6) + parser.add_argument("--n-alpha", type=int, default=5) + parser.add_argument("--alpha-min", type=float, default=0.1) + parser.add_argument("--alpha-max", type=float, default=0.9) + parser.add_argument("--n-trials", type=int, default=10) + parser.add_argument("--n-baseline", type=int, default=24) + parser.add_argument("--seed", type=int, default=20260506) + parser.add_argument("--observable-set", choices=["rank_audit", "scale_0330"], default="rank_audit") + parser.add_argument("--out", default="tools/data/perturbation_dimensionality_audit.json") + args = parser.parse_args() + OBSERVABLE_SET = args.observable_set + run(args) + + +if __name__ == "__main__": + main() tokens used 110,385 Eseguito esperimento e scritto il report richiesto: [tools/data/reports/agent_20260506_0625.md](/opt/MM_D-ND/tools/data/reports/agent_20260506_0625.md) Output creati: - [tools/exp_perturbation_dimensionality_audit.py](/opt/MM_D-ND/tools/exp_perturbation_dimensionality_audit.py) - [tools/data/perturbation_dimensionality_audit.json](/opt/MM_D-ND/tools/data/perturbation_dimensionality_audit.json) - [tools/data/perturbation_dimensionality_audit_scale0330.json](/opt/MM_D-ND/tools/data/perturbation_dimensionality_audit_scale0330.json) Risultante: il claim forte “GUE ha un secondo asse di perturbazione stabile” non sopravvive così com’è. Su repliche GUE lunghe il secondo asse scende a PC2 ~9.9% nel set osservabile del run 03:30; i campioni GUE corti gonfiano il rank. Emerso anche un vincolo META: `SR`, `SR2`, `triple_var` non indicano le stesse funzioni tra gli script recenti. Aggiornato anche [tools/data/seme.json](/opt/MM_D-ND/tools/data/seme.json) con il vincolo. Verifica: `python -m py_compile tools/exp_perturbation_dimensionality_audit.py` passata.