Measurable Creative Artificial Intelligence

Exploring the possibilities, methods, and importance of measuring creativity.

Workshops | Organizers | Publications

Workshops

SIGGRAPH - Measurable Creative AI

August 11, 2021 at the Special Interest Group on Computer Graphics and Interactive Techniques Conference

The idea of applying AI to creative endeavors activates the games/vfx industries, artists, manufacturing, tech company labs, and academics, but much of what is published on the topic shows there is still a great degree of siloing within these various fields. SIGGRAPH is an ideal context for these diverse constituencies to develop shared perspectives on goals, outcomes, and metrics for this emerging field.

This half-day workshop for SIGGRAPH pulls from the industries above to give a wide view on what "creative AI" means in practice and what people's goals and aspirations are in the space. Framed with panel discussion from field experts, we will conduct a hands-on session around "measurable" goals to collaboratively define, apply, and iterate on metrics for evaluating creative AI.

This workshop aims to bring people together to talk about common goals (the "Measurable" part of the workshop title) and objectives in this space as well as inform the direction that researchers take when developing creative AI tools, and potentially form the foundation for new collaborative relationships.

CVPR - Bridging the Gap between Subjective and Computational Measurements of Machine Creativity

June 20, 2021 at the Conference on Computer Vision and Pattern Recognition
While the methods for producing machine creativity have significantly improved, the discussion on a scientific consensus on measuring the creative abilities of machines has only begun. As Artificial Intelligence becomes capable of solving more abstract and advanced problems (e.g., image synthesis, cross-modal translations), how do we measure the creative performance of a machine? In the world of visual art, subjective evaluations of creativity have been discussed at length. In the CVPR community, by comparison, evaluating a creative method has not been as systematized. Our goal in this workshop is to discuss current methods for measuring creativity both from experts in creative artificial intelligence as well as artists. We do not wish to narrow the gap between how humans evaluate creativity and how machines do, instead we wish to understand the differences and create links between the two such that our machine creativity methods improve.

ISEA - Collaboratively Designing Metrics to Evaluate Creative Machines

October 18, 2020 at the International Symposium on Electronic Art
How do we make a creative machine? This workshop follows a series of studies conducted in the classroom setting at CMU and UC San Diego. For ISEA, participants will collectively find a definition of creativity specific to machines, addressing follow up questions: Can we computationally model ambiguity? Would a novelty search result in valuable discoveries? Where is the threshold between randomness and creativity? How do we evaluate the creativity of an algorithm? To answer these questions, participants develop sets of criteria to assess their own and peer groups’ creative AI and ML projects. Although such a human-centered method is subjective, we anticipate discovering ways to describe and interpret dimensions of algorithmic creativity. This half-day workshop engages a broader arts and machine learning community to collaboratively define these quantitative criteria, in a first attempt to collectively establish evaluation metrics for the area of creative AI.

Organizers

Eunsu Kang is an artist, a researcher, and an educator who explores the intersection of art and machine learning as well as the possibility of creative AI. She started her artist career with video installations and single channel videos. She left her tenured art professor position to design and teach new courses (Art and Machine Learning, Creative AI) at the Machine Learning Department of Carnegie Mellon University. Recently she co-founded the Women+ Art AI collective.

Jean Oh is a faculty member at the Robotics Institute at Carnegie Mellon University. She is passionate about creating robots that can collaborate with humans in shared or remote environments, continuously improving themselves through learning, exploration, and interactions. Jean co-designed a new graduate-level course on Creative AI at CMU and was a co-organizer of the first workshop on Measuring Computational Creativity at ISEA’20.

Robert Twomey is an Assistant Professor of Emerging Media Arts at the University of Nebraska-Lincoln, and a Visiting Scholar with the Clarke Center for Human Imagination at UC San Diego. His work as an artist and engineer explores how emerging technologies transform sites of intimate life. He has presented his work at SIGGRAPH (Best Paper Award), the Museum of Contemporary Art San Diego, and has been supported by the National Science Foundation, the California Arts Council, Microsoft, Amazon, and NVIDIA.

Peter Schaldenbrand is a graduate student and technical staff member at Carnegie Mellon University. His research interests include creating machine learning models that perform creative tasks and artificial intelligence in education. Recently, he has been focusing on a robot artistic painting project.

Ahmed Elgammal is a professor at the Department of Computer Science at Rutgers University. His research areas include data science in the domain of digital humanities. His work on knowledge discovery in art history and AI art generation received wide international media attention, and his art has been shown in several technology and art venues in Los Angeles, Frankfurt, San Francisco, and New York City.

Hyeju Jang is a postdoctoral fellow at the University of British Columbia. Her research interests include natural language processing, computational linguistics, discourse analysis, and text mining in various domains. She has been working on computationally modeling creative uses of language, such as metaphor, in order to capture how they are used in discourse context and identify a broader spectrum of predictors that contribute towards their detection and generation.

James McCann is an Assistant Professor in the Carnegie Mellon Robotics Institute. He is interested in systems and interfaces that operate in real-time and build user intuition, including systems that enable and enhance creativity.

Devi Parikh is an Associate Professor in the School of Interactive Computing at Georgia Tech, and a Research Scientist at Facebook AI Research (FAIR). Her research interests are in computer vision, natural language processing, embodied AI, human-AI collaboration, and AI for creativity. Devi has co-organized workshops at CVPR since 2010. For instance, the series of Visual Question Answering workshops.

Jun-Yan Zhu is an Assistant Professor in the School of Computer Science of Carnegie Mellon University. He studies computer vision, computer graphics, computational photography, and machine learning, with the goal of building intelligent machines, capable of recreating our visual world. Jun-Yan has co-organized several relevant workshops and tutorials including CVPR 2020 Tutorial on Neural Rendering, ICCV 2019 Workshop on Image and Video Synthesis, and CVPR 2018 Tutorial on Generative Adversarial Networks.

Publications

Coming soon