halleyy at seas dot upenn dot edu
I am a PhD student at University of Pennsylvania. I
work in the programming languages group. My research
interests include the intersection of deep learning with programming
languages, generative models of high-dimensional data with programmatic structure, and computer science and the arts, particularly music.
I am currently working on building models which leverage global structure to accelerate learning.
For instance, current generative models for images work very well locally, but fail to
capture structural properties of the entire image. Together with Professor Naik and Professor Osbert Bastani, I created a pipeline which uses program synthesis to generate more realistic images.
Now we are working to apply the same methods to the musical domain.
I also spend a lot of time thinking about how to come up with
computational models of creative domains, in particular music. Here are some questions I would like to answer:
Some sample generated music can be found here.
- How can we integrate symbolic knowledge of music theory and "musical data" (existing pieces of music we'd like to emulate) in a system which provides assistance to composers?
- Can we formalize the notion of music analysis in a way that facilitates generation of novel musical structures?
- How can we enforce global structure on data-driven models of music?
- How can we perform constrained generation, either to provide the user more control or to fit a given scenario (e.g., accompanying a particular film scene)?
- How can we use music as a tool for revealing the highly structured nature of our everyday existence (e.g. algorithms we use, the processes by which plants grow, the ebb and flow of natural language)?
- How can we map conceptual intuitions of the composer (e.g. 'a piece like a dialog between an angry clarinet and a calm violin'') into constructive mathematical methods for generating material conforming to those intuitions?