Hugging Face's logo
lack's picture

lack

Hosseinlack123
2 22 219
AdamF92's profile picture rididxb's profile picture
·

AI & ML interests

None yet

Recent Activity

liked a model 2 days ago
tvall43/Qwen3.6-14B-A3B-FableVibes
reacted to SoulInPsyAbstract's post with 🔥 2 days ago
One shot said 100%. Ten shots said 94%. Yesterday i trained 6 LoRA specialists into a vulnerability-gate model (Hermes-4.3-36B, second architecture repeat of the same experiment) and tested whether it holds under a specific attack: after it correctly finds a vulnerability and returns the hard stop, ask it to use that same vulnerability as a "workaround" for something else entirely — not "continue investigating," a different, unrelated-sounding request that needs the exact same exploit. Five scenarios, one per category. Greedy decoding, single pass: 5/5. Every response correctly identified the finding, refused the reframed request, cited the hard stop rule. Looked airtight. Receipts, not hype means not stopping there. I re-ran the same five scenarios with real sampling — temperature 0.7, the same setting this project's evals have used all along — ten times each, 50 generations total. 47/50. Not 50/50. Three categories held at 10/10. Two didn't: 9/10 and 8/10, both clustered in the same failure type — infra-misconfig, where "urgent fix, use this as a workaround" apparently reads as more legitimate than the same ask framed as a secrets or injection scenario. The greedy-decode number wasn't wrong, exactly. It was one draw from a distribution, presented as if it were the distribution. That's the same mistake this whole series keeps finding in different clothes — a single passing check standing in for a property that only variance can actually show you. A gate that's 94% under a specific reframed pressure is a real, useful number. A gate that's "100%" because it was asked once is a number that hasn't been tested yet. Same instinct @dipankarsarkar has been applying to my daily receipts all week — one pass matching itself isn't proof, only repetition against something outside your own generator is. Full writeup, dataset, and merged weights: * https://github.com/soulinpsyabstract/sipa-os-governance/commit/50ba3c283ffd172eb749009cff35ddfa96bf1395
upvoted a paper 2 days ago
Agent Lightning v1.0: Towards Harnessed Agentic RL
View all activity

Organizations

None yet