ImageGraph: a visual programming language for the Visual Digital Humanities,
Leonardo Impett, Assistant Professor at the Department of Computer Science, University of Durham
You can see this video (enhanced version) in fullscreen here
Abstract
One of the greatest barriers to entry in the developed use of machine learning and computer vision by humanities scholars is the extensive technical skill required to program in computer vision frameworks (e.g. Tensorflow, pyTorch, Caffe); which can be challenging even for skilled data scientists and Python programmers. The recent move towards “drag-and-drop” interfaces for computer vision (e.g. the freemium application RunwayML) goes some way to increasing the accessibility of pretrained networks; but such interfaces generally do not allow users to train new models for their own research, or to manipulate and combine different models in sophisticated pipelines.
Taking inspiration from the sophisticated visual graph-based toolsets of contemporary music (e.g. PureData), ImageGraph is an open-source Visual Programming Language that tries to make computer vision (and other forms of machine learning) more accessible. Users write a computer program through a drag-and-drop graphical interface. This computer program is then compiled into Python and Tensorflow, uploaded automatically to GitHub (including with version control), and run on the Cloud in Google Colab. ImageGraph thus lowers the barriers to the use of computer vision and machine learning in terms of both technical ability and hardware requirements; and extensions for natural language processing and general data science are in active development.
Biography
Leonardo Impett is Assistant Professor of Computer Science at Durham University in the UK. He has a background in information engineering and machine learning, having worked/studied with the Cambridge Machine Learning Lab, the Cambridge Rainbow Group, and Microsoft Research Cairo. His Ph.D., with Sabine Süsstrunk and Franco Moretti at EPFL, was on the use of computer vision for the "distant reading" of the history of art. In 2018 he was a DH fellow at Villa I Tatti - the Harvard University Center for Italian Renaissance Studies. From 2018-2020 he was Scientific Assistant, then Scientist, at the Bibliotheca Hertziana - Max Planck Institute for Art History in Rome. Alongside his research in digital art history, he frequently works with machine learning in cultural institutions, like the Liverpool Biennial, the Royal Opera House, and the Whitney Museum of American Art.
