Sensorify
“Sensorify v1.0” exhibited at Tension, MOMus Museum of Contemporary Arts, Thessaloniki, Greece.
The theme of Saphire’s work revolves around a future scenario in which human communication takes place entirely through the internet. This communication expands and includes human sensations and emotions that will be digitally communicated to their recipient.
The theme reflects the dystopian manifestation of the future as physical contact will have been replaced by digitized data.
The installation presents the digitization of emotions as they are recorded in brain activity through an EEG. The digital presentation of the emotions will be projected on one screen while the digitization process takes place on the other.
Photogrammetry is the science of making measurements from photographs, typically used to map the surface of a physical object or environment. In Saphire’s installation, this is achieved using the software Colmap. A Sony a6400 4k Camera is used to capture a collection of high-resolution photographs from multiple angles of a subject’s face, mapping every contour, wrinkle, and feature. These images are then processed using photogrammetry to create a three-dimensional point cloud, a dense cluster of data points that replicates the unique physical landscape of the face in digital space. This point cloud serves as a visual representation of the subject, which can be digitally manipulated and visualized, making it an integral part of the installation’s exploration of digitized human emotions and interaction.
TouchDesigner, a visual development platform, plays a crucial role in manipulating these three-dimensional point clouds generated through photogrammetry. TouchDesigner is renowned for its capability to handle real-time 3D rendering and high-level interactive control, making it the perfect tool to take the complex data from the point cloud and transform it into an interactive, visually compelling form. With the data from the EEG headset, TouchDesigner is used to apply changes to the point cloud in real-time, reflecting the subject’s changing emotions and reactions. The resulting visualizations are then projected on the screen, offering a unique and immediate view into the digitized representation of human emotion. This interplay between the physical and digital, made possible by the technology of photogrammetry and the capabilities of TouchDesigner, is what gives the artwork its profound impact.
Software // Colmap (photogrammetry), TouchDesigner, Adobe Photoshop, Adobe Premiere, Emotiv Brain Computer Interface.
Hardware // Emotiv Insight (EEG Headset), Sony a6400 4k Camera, Xbox Controller (audience interaction).
After the success of our installation at the exhibition of MOMus Museum of Contemporary Art, we’re re-designing the installation to utilize facial expression recognition, allowing the machine to better understand our behaviors and adapt to personal sensations. While some may view this as a departure from human connection, it’s clear that social media has already begun to replace physical interaction in our lives. Machines are becoming better equipped to understand our behaviors and provide personalized experiences, leaving us increasingly reliant on technology and digital interactions.
This artwork is now moving into its second phase, with the incorporation of Sensorify v2.0. This version expands on the installation’s capabilities by introducing facial expression recognition. It utilizes Reactjs for building the user interface, Threejs for 3D modeling, and face-api for facial recognition. This upgrade is designed to enhance the machine’s understanding of human behavior and offer a more personalized emotional digitization experience.
As this software upgrade may be seen as a further departure from human connection, it’s important to consider that social media and technology have already begun replacing physical interaction in our daily lives. This installation challenges us to reflect on our increasing dependence on digital communication and consider the implications of a future where our emotions are communicated not through physical contact but through digital data.
Saphire’s work is a striking comment on the potential for technology to change the way we connect, communicate, and understand each other. As the technology continues to evolve, this installation is sure to stimulate further thought and conversation about the future of human interaction.