[I want to dance]: Comparative K-Pop Choreography Analysis through Deep-Learning Pose Estimation,
Timothy R. Tangherlini, Professor, Department of Scandinavian at University of California, Berkeley; Peter Broadwell, Stanford University
You can see this video (enhanced version) in fullscreen here
Abstract
Critics have long noted the strong visual aspects of K-pop, with the videos for newly released songs garnering millions of hits in a very short time span. A key feature of many Kpop videos is the dancing. Although many of the official videos are not solely dance focused, incorporating aspects of visual storytelling, nearly all of Kpop videos include some form of dance. In addition to the "main" video for a Kpop release, the release of a dance video, or a dance rehearsal video, focusing exclusively on the dances has become common practice. These videos allow fans to learn and practice the dance, thereby increasing the kinesthetic connection between fans and their idols.
At the same time, it affords an opportunity to explore the "dance vocabulary" of Kpop dances. While there are well-known Kpop choreographers who work with the Kpop idols to create their dances, there is little documentation of these dances beyond the dance videos themselves. In our work, we develop a series of methods for (a) identifying dance sequences in Kpop videos, irrespective of whether they are dance videos (b) develop a series of classifiers for the navigation of a large scale Kpop video corpus and (c) apply deep learning methods to identify dancers and their body positions.
Taken together, these approaches pave the way for the development of a macroscope for the study of Kpop videos, allowing researchers to identify patterns in the Kpop space, explore dynamic change in features such as color space, or interrogate the differences in visual representations of male and female performers at an aggregate scale. Importantly, as pose estimation has become more accurate, these methods allow us to begin the process of inferring the dance vocabulary of Kpop and start the process of tracing transcultural choreographic flows.
Biography
Timothy R. Tangherlini is a Professor in the Dept. of Scandinavian at the University of California, Berkeley. A folklorist and ethnographer by training, he is the author of Danish Folktales, Legends and Other Stories (University of Washington Press, 2014), Talking Trauma (University Press of Mississippi, 1999), and Interpreting Legend (Garland Publishing, 1994). He has published widely in academic journals across the fields of folklore, sociology, and computer science. He also acted as co-director of the program on Culture Analytics at the NSF’s Institute for Pure and Applied Mathematics. He is interested in the circulation of stories on and across social networks, and the ways in which stories are used by individuals in their negotiation of ideology with the groups to which they belong. His current research focuses on computational approaches to problems in the study of folklore, literature and culture. His research has been supported by grants from the NSF, the NIH, the NEH, the ACLS, the Guggenheim Foundation, and Google. He is a fellow of the American Folklore Society, and of the Swedish Royal Gustav Adolf Academy.
Peter Broadwell is a Digital Scholarship Research Developer at the Stanford University Libraries’ Center for Interdisciplinary Digital Research, where his work applies machine learning, web visualization, and other methods of digital analysis to complex cultural data. He has a Ph.D. in Musicology from the University of California, Los Angeles and an M.S. degree in Computer Science from the University of California, Berkeley. Prior to joining the Stanford Libraries, he was an Academic Projects Developer in the UCLA Digital Library program, where he contributed to work investigating the relationships between social media, television news, and web content. Recent studies in which he has participated have involved automatic translation and indexing of folklore collections in multiple languages, as well as intermedia analysis of Japanese Noh theater performances.
