Mienophone
54*28*22cm, glass, multi-media installation, Zhé Wang & Daniel Heitz, 2021，Migrant Bird Space, Berlin, Germany
表情声音
玻璃，多媒体装置，54*28*22cm，王哲&Daniel Heitz，2021，候鸟空间，柏林，德国
A digital synthesizer using facial expressions to modulate sounds.
Music is controlling our emotions. It sometimes feels a bit like being a string puppet, while music is
the puppet master. It can make us dance or sink into sadness.
As Machine Learning Algorithm is becoming better at reading human emotions, can we use this technology to reverse this power of control? How does music sound like if it was directly controlled by our expressions and not the other way around? How would we react? Would it perhaps reinforce the emotion? Can technology help us to become more aware and better understand our emotions? And how does all this challenge the purpose of art and the role of the artist in the future?
Mienophone is created by Zhe Wang and Daniel Heitz in collaboration, since 2019.
Funded by KARIN ABT-STRAUBINGER Stiftung.
表情声音是艺术家王哲和Daniel Heitz自2019年起一起合作的一个艺术项目。
音乐往往可以控制我们的情绪。我们在听音乐时，我们的情绪就像是提线木偶，而音乐就是我们的主人。它可以轻意使我们跟着节奏摇摆或是陷入悲伤。
随着机器学习算法在阅读人类情绪方面能力的提高，我们可以使用这项技术来逆转这种控制吗? 如 果我们可以通过表情来控制声音，那听起来会如何? 我们又会对这种声音作出什么反应? 这会增强 我们的情绪吗? 未来技术是否可以帮助我们变得更加了解并更好地理解我们的情绪? 所有这些会如何挑战艺术的创造力和艺术家的未来?
这些是我们想通过Mienophone探讨的问题。
项目由KARIN ABT-STRAUBINGER Stiftung 基金会支持
技术支持 Berlin Glassworks
Mienophone is an installation consisting of glass sculpture and a software synthesizer developed in MAX, a visual programming language for music and multimedia. A camera is recording a live video stream of the viewer. Selected frames of the video live stream are transmitted to the Microsoft Cognitive Services Vision API.
The Machine Learning Algorithm returns a numeric value for a set of emotions detected in the player's face in the video. The emotions detected are anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. Each value is used to control a set of parameters of the software synthesizer and allows the viewer to influence the sound by expressing emotions.
The camera is placed in an organic-shaped glass sculpture. Since the glass surface is not flat, the camera's line of sight will be distorted, so the Machine Learning Algorithm will also read the viewer's image distorted. This also reminds us that whether Machine Learning Algorithm can see us clearly, does AI and humans really see each other? Is the "seeing" equal?
The shape of the glass is symbolically presented in a form similar to a human face; just like our imagination of AI, it is a human-like, inhuman subject. When faced with this infant-sized, fragile, human-like device, we will regard ourselves as parents of this new intelligence.
The main element that composes glass is silicide, and computer chips are the same. The history of glassmaking can be traced back to BC, far before the invention of iron melting technology. Although computer chips have only be invented for a few decades, the powerful driving force they provide to the development of human society cannot be quantified.
The development of Machine Learning Algorithm has been closely linked to the future of humankind. In the most optimistic scenario, we may be able to create a utopia where humans and superintelligence can coexist peacefully. Or, the omnipotent AI has mastered higher intelligence than human beings. Human beings live in a society controlled by AI, lose their creative ability, and only lament their own destiny.
Mienophone是由玻璃雕塑和软件合成器组成的装置，该合成器是在MAX(一种用于音乐和多媒体的视觉编程语言)中开发的。装置中的相机会实时录制观看者的表情，然后选定帧将被传输到Microsoft Cognitive Services Vision API。 由机器学习驱动的算法会为实时录制的视频中检测到的一组面部表情返回一组数值。检测到的数据对应的情绪可能是愤怒，轻蔑，厌恶，恐惧，快乐，平静，悲伤或惊讶。每个数值会用于控制软件合成器的一组参数，从而允许观看者通过表达情绪来影响合成器的声音。
相机被放置在有机形状的玻璃雕塑中。由于玻璃表面不平整，相机的视线会发生扭曲，因此AI也会看到观看者的形象发生扭曲。这也提醒我们，即人工智能是否能够清晰地看到我们，人工智能与人类的对视是否会存在? 是否平等?
玻璃的形状象征性的以类似于人脸的形状呈现，就好像我们对于人工智能的想象一样， 是一个似人非人的主体。在面对这个如婴儿大小， 脆弱，似人的装置时， 我们会将自己视为这种新智能的父母。
组成玻璃的主要元素是硅化物，而制造计算机芯片其实是相同的材料。人类制作玻璃的历史可以追溯到公元前，远远早于熔铁技术的发明。而计算机芯片虽然只出现了几十年，但它对人类社会的发展所提供的强大驱动力是无法量化的。而在机器学习基础上发展的人工智能也越来越不可控了。
如今人工智能的发展已经和人类的未来紧密相连了。最乐观的情况，我们也许能够创建一个乌托邦，在那里人类与超级智能可以和平共处。又或者，无所不能的AI掌握了高于人类的智能，人类生存于受AI掌控的社会，失去了创造能力，只能为自己的命运哀叹。