cv

Master

University of Cambridge

AI Ethics and Society

• Machine Learning Alignment
• AI Ethics
• AI Governance
• CS230 Deep Learning
• ML Safety
• Discrete Mathematics
• CS234 Reinforcement Learning
• CS109 Probability for Computer Scientists
• Mechanistic Interpretability
• Algorithms and Data Structure

Summer Student

Stanford University

Cognitive Science & Philosophy

• Mathematics Foundation of Computing
• Minds and Machines
• Introduction to Neuroscience

Bachelor of Arts

Shantou University

English & Global Studies

• Machine Learning and relevant maths
• Research Methodology
• Linguistics

Position: AI Presents Catastrophic Epistemic Risks

We argue that AI presents catastrophic epistemic risks, which deserves a closer look and more research attention to.

• Position: Co-author
• Collaborators: Kellin Pelrine, Dan Zhao, Tianyi (Alex) Qiu

Stay True to the Evidence:Measuring Belief Entrenchment in LLM Reasoning via the Martingale Score

We propose an alternative to RLHF that uses " helping humans to seek truth as training objective " and human opinion change data as ground truth. By doing so we aim to remove feedback loop-incurred lock-in from its root and revolves alignment evaluation around " LLM-assited human performanceWe ".

• Position: Project Co-founder
• Collaborators: Prof Maarten Sap (CMU), Prof Hirokazu Shirado, Tianyi (Alex) Qiu (CHAI, Berkley & PKU

Open Problems in AI Influence

We propose "AI influence " as a field of studies on AI's impact on epistemics and morality. In this paper we collect open problems in AI influence, as long as methodologies from ML, computational social science, cognitive science to study AI influence.

• Position: Project Co-founder
• Collaborators: Prof Max Kleiman-Weiner (UW), Tianyi (Alex) Qiu (CHAI, Berkley & PKU, Prof Atoosa Kasirzadeh, Prof John P Wihbey, Dr Moshe Glickman, Tao Lin.

The Lock-in Hypothesis: Stagnation by Algorithms

We are concerned with the problems of LLM-incured value lock-in and knowledge collapse (as probable as model collapse since increasingly our discourses are mediated by AI systems and iterative training becomes more prevalence), with the consequence being more destructive. Our team establishes real-world evdience of lock-in from WildChat data and builds simulations and formal modeling of the mechanisms of LLM-incurred value lock-in

• Position: Project Co-founder
• Collaborators: Prof Max Kleiman-Weiner (UW), Tianyi (Alex) Qiu (CHAI, Berkley & PKU

Multilevel analytical framework for interpretability

Research on cognitive science and neuroscience to address interpretability challenges in ML.

• Position: Project Lead
• Goal: Publication in Transactional Machine Learning Research
• Senior authors: Grace W. Lindsay(NYU), Prof Anna Ivanova (GeorgiaTech)

Comprehensive Survey on AI Alignment

Survey paper on alignment research for newcomers.

• Focus: Interpretability challenges in ML
• Collaborators: Yaodong Yang, Jiaming Ji, Tianyi Qiu

Harms from agentic algorithmic systems

Research on safety and harms from agentic systems in AI.

• Highlight: Published paper cited by GPT-4 and high-profile AI safety reports

Stanford Existential Risks Initiative (SERI)

Research on China's AI governance approach.

• Position: Research Fellow

Open Philanthropy Graduate Scholarship

Open Philanthropy

A full scholarship for the master of study in AI Ethics and Society at the University of Cambridge.

Manifund AI Research Scholarship

Manifund

A research scholarship to support work on neuroscience and mechanistic interpretability.

Hong Kong Cyberport Creative Micro Funding

Hong Kong Cyberport Management Company Limited, wholly owned by the Hong Kong SAR Government

Early-stage startup funding to support work on Homeal, a sharing economy app about food and culture.