Warren Center Affiliate Profile- Annie Liang

Novel Uses of “Big Data”

Developments in the use of “big data” have raised new questions for society. Recent work in computer science, law, sociology, and other disciplines has highlighted potential harms such as lack of privacy, lack of accountability, and bias in machine learning. But there are additionally other, economic, consequences that we do not yet understand. Big data and use of machine learning algorithms constitute a large change in the kinds of information and prediction algorithms that are used in real incentive systems, which have largely not been studied yet from the perspective of economic theory. Annie Liang, an Assistant Professor of Economics, is working towards answering these questions.

Annie’s theoretical research focuses on informational environments with complex data–for example, firms in which data scientists have access to a large number of different data sources, some of which complement each other. In a series of papers, Annie has studied both the optimal use of such data and the inefficiencies that are likely to arise in practice. In new work, Annie is modeling emerging data markets, in particular, how novel uses of ‘big data’ reshape basic economic incentives, and what implications this has for consumer and social welfare.

There remain many open economic questions around the production and use of data, towards which formal frameworks can provide useful structure and insight. In the future, Annie is excited to continue working with other researchers working on topics related to information and data, including computer scientists and statisticians.

Using Machine Learning to Improve Economic Modeling

Annie’s other main area of research focuses on how techniques in machine learning can be applied to evaluate and build economic models. For example, an old question in game theory is how people behave in new strategic situations. One of Annie’s papers considers the problem of predicting initial play in games and demonstrates how machine learning algorithms can be used to uncover new, interpretable regularities in people’s behavior, and to develop practical extensions of existing models.

Another paper examines how to assess how well a given model does in prediction. This evaluation is difficult because the same objective level of predictive accuracy can mean very different things in different problems. Predictive limits differ across problems depending on the outcome being predicted, and the feature set that is available. Thus, to interpret the predictive accuracy of a given model, we need to compare it not against perfect prediction but against the best achievable level of prediction in a given problem. By using methods based on machine learning, Annie works to construct a benchmark for the evaluation of the predictive accuracy of existing theories.

Annie’s Background

Annie received her Ph.D. in Economics in 2016 from Harvard University and was previously a Postdoctoral Researcher at Microsoft Research New England before joining Penn’s faculty in 2017.

Her introduction to game theory was through a class with Glenn Ellison as an undergraduate student at MIT. “This was the first class I’d taken in which the material was presented not as fact but as a set of still evolving perspectives,” says Annie. Looking back, what she found special was that economic theory was still young enough to have open fundamental questions.

Annie has taught several undergraduate and graduate courses over the past few semesters including Game Theory, Topics in Economic Theory: Beliefs and Learning, and Topics in Economic Theory: Learning, Information, and Data. In addition to teaching, she has served on numerous program committees, refereed economics journals, and organized several conferences.

In her free time, Annie works on her Russian language skills with a tutor. She started learning Russian about seven years ago, and loves the language. “For me,” says Annie, “there is a strong analogy between language development and research, since both involve gradual steps towards a long-term goal.”

Annie is also a hobbyist figure skater who enjoys skating at the Penn Ice Rink, and a huge fan of HBO’s Game of Thrones series.