William Merrill

headshot.jpg

I am a Ph.D. student at the CDS at NYU, where I am advised by Tal Linzen and supported by an NSF graduate research fellowship and by AI2.

My research develops theory to better understand what language models can do, as well as what they can't. I've worked on characterizing the computational power of transformers for representing linguistic structure and solving reasoning problems. I've also analyzed the aspects of semantics that can be learned from co-occurrence patterns as a way to understand the potential of self-supervised learning.

Contact: willm[æt]nyu.edu or here for anonymous feedback

Outside of research, I like exploring New York City by foot, train, and boat. I like cooking new things and trying hole-in-the-wall restaurants. I also play basketball, ping pong, and Age of Empires II.



Latest posts

Publications

2023

  1. DLT
    Formal Languages and the NLP Black Box
    William Merrill
    In Developments in Language Theory, Jun 2023
  2. ME-FoMo
    A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks
    William Merrill, Nikolaos Tsilivis, and Aman Shukla
    In ICLR Workshop on Mathematical and Empirical Understanding of Foundation Models, Jun 2023
  3. TACL
    Transparency Helps Reveal When Language Models Learn Meaning
    Zhaofeng Wu, William Merrill, Hao Peng, and 2 more authors
    TACL, Jun 2023
  4. How Language Model Hallucinations Can Snowball
    Muru Zhang, Ofir Press, William Merrill, and 2 more authors
    Jun 2023
  5. NeurIPS
    A Logic for Expressing Log-Precision Transformers
    William Merrill, and Ashish Sabharwal
    In NeurIPS, Dec 2023
  6. TACL
    The Parallelism Tradeoff: Limitations of Log-Precision Transformers
    William Merrill, and Ashish Sabharwal
    TACL, Jun 2023

2022

  1. CoNLL
    Entailment Semantics Can Be Extracted from an Ideal Language Model
    William Merrill, Alex Warstadt, and Tal Linzen
    In CoNLL, Dec 2022
  2. Extracting Finite Automata from RNNs Using State Merging
    William Merrill, and Nikolaos Tsilivis
    Jan 2022
  3. TACL
    Saturated Transformers are Constant-Depth Threshold Circuits
    William Merrill, Ashish Sabharwal, and Noah A. Smith
    TACL, Aug 2022
  4. ACL
    ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
    Sanjay Subramanian, William Merrill, Trevor Darrell, and 3 more authors
    In ACL, May 2022

2021

  1. EMNLP
    Competency Problems: On Finding and Removing Artifacts in Language Data
    Matt Gardner, William Merrill, Jesse Dodge, and 4 more authors
    In EMNLP, Nov 2021
  2. TACL
    Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?
    William Merrill, Yoav Goldberg, Roy Schwartz, and 1 more author
    TACL, Sep 2021
  3. EMNLP
    Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent
    William Merrill, Vivek Ramanujan, Yoav Goldberg, and 2 more authors
    In EMNLP, Nov 2021

2020

  1. ACL
    A Formal Hierarchy of RNN Architectures
    William Merrill, Gail Weiss, Yoav Goldberg, and 3 more authors
    In ACL, Jul 2020
  2. COVID19
    CORD-19: The COVID-19 Open Research Dataset
    Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, and 25 more authors
    In ACL Workshop on NLP for COVID-19, Jul 2020
  3. On the Linguistic Capacity of Real-Time Counter Automata
    William Merrill
    Sep 2020

2019

  1. DeLeFoL
    Sequential Neural Networks as Automata
    William Merrill
    In ACL Workshop on Deep Learning and Formal Languages, Aug 2019
  2. BlackboxNLP
    Finding Hierarchical Structure in Neural Stacks Using Unsupervised Parsing
    William Merrill, Lenny Khazan, Noah Amsel, and 3 more authors
    In ACL Workshop BlackboxNLP, Aug 2019
  3. LChange
    Detecting Syntactic Change Using a Neural Part-of-Speech Tagger
    William Merrill, Gigi Stark, and Robert Frank
    In ACL Workshop on Computational Approaches to Historical Language Change, Aug 2019

2018

  1. BlackboxNLP
    Context-Free transductions with neural stacks
    Yiding Hao, William Merrill, Dana Angluin, and 4 more authors
    In EMNLP Workshop BlackboxNLP, Nov 2018
  2. NAACL
    End-to-End Graph-Based TAG Parsing with Neural Networks
    Jungo Kasai, Robert Frank, Pauli Xu, and 2 more authors
    In NAACL, Nov 2018
  3. TULCon
    A semantics of subordinate clauses using delayed evaluation
    William Merrill
    Toronto Undergraduate Linguistics Conference, Nov 2018