Abstract. I work at the intersection of AI research and production engineering. My published work examines fragility in benchmarking and proposes filtering methods that increase robustness. Beyond evaluation, I have worked on synthetic data generation, AI-generated text detection, and dataset augmentation for energy-efficient inference on custom silicon. In production, I build the ML systems that serve these insights at scale: notification ranking and delivery infrastructure, recommendation pipelines, and open-source LLM benchmarking platforms. I am interested in closing the loop between empirical research and deployed systems, particularly for non-verifiable generative tasks where no objective ground truth exists.
1 Research Interests
LLM benchmark reliability and dataset quality; AI-generated text detection; evaluation of non-verifiable generative tasks; artifact-centric quality metrics; human judgment and behavioral estimation in AI systems.
Expert gated artifacts as evaluation criterion for non-verifiable tasks
Any session which iteratively refines a shared artifact-code, a contract, an email draft-can be decomposed into ordered layers, each consisting of a prompt, a responder’s modification, and an optional direct edit by the prompter. The final accepted artifact serves as ground truth, and quality at each layer is measured by edit distance to that terminal state. Crucially, ground truth is defined as expert-accepted state rather than objective optimality: the prompter need only recognize progress, not know the ideal output in advance. This makes the framework applicable to any generative task with no unique correct answer, and lets it produce cross-responder comparable scores, convergence metrics, and cost-of-correction signals from production sessions that would otherwise go unmeasured.
Doomer writings give fodder for models to be misaligned on.
One of my biggest fears with AI safety is that we are creating doomsday scenarios which the models train on. It is not hard to imagine that with information on exactly how to cause a crisis as an LLM, the LLM becomes more likely to cause that crisis. Additionally, writings presupposing problematic behavior like blackmailing for an LLM may make the models more likely to associate themselves with blackmailing. Therefore, it may ultimately be better to be optimistic about AI safety-at least in writing.
LLMs and lived experience
The tasks that gate expertise-taste, strategy under uncertainty, design, diagnosis-are precisely the ones for which no cheap correctness signal exists. A model trained on the corpus centroid cannot be steered all the way to a specific person’s lived judgment. The right move is not to make the model more expert, but to borrow a human’s opinionation via behavioral estimation while handing verifiable work to the optimizer.