Fourth Wall. Unconscious traces

•

Interactive Installation concept and design by Tommaso Bertagnin, Laura Panno, Helena Principato

February - June 2013

The open-air “atrium” of San Giovanni Evangelista in Venice is by day a busy thoroughfare. Visitors not in a hurry stand to admire its emphatic architecture, or use it to pose for photos.

Fourth Wall installs, high on the atrium’s flagpole, a depth camera which during the day records people moving and gesturing in the space. Also on the flagpole is a projector.

At night, when San Giovanni is dark and almost deserted, you can use Fourth Wall’s smartphone app to control a torch-like beam from the projector. Scanning the pavement with this beam exposes the moving outlines of the day’s recorded visitors, as and where they were a few hours before. These computer-generated “shadows” produce an outline with enough detail to indicate the person’s individual characteristics and behaviour, but not enough to invade their privacy or dispel the sense of haunted mystery.

Thus Fourth Wall transforms the portal of San Giovanni Evangelista into a stage, where people that pass by in great number every day, can be seen at night through their shadows.

Fourth Wall highlights the invisible barrier that lies between people, by revealing behaviours and allowing people’s actions to be explored more intimately.

Design and Prototype

A compact projector and depth camera are fixed at the top of the red flagpole in front of the portal. So the installation is unobtrusive, hardly affecting the place physically. Those who pass through by day don’t realize that something will happen at night. This is because the project aims to capture the natural behaviour of people who don’t know how or when they are being recorded. Only at night do they realize they are part of an installation.

Interaction

The installation is activated by the presence of nighttime visitors in the square. A QR Code is then projected close to them, and the interaction starts when it is connected to the Fourth Wall web application. They can interact with the installation by simple gestures on their smartphone screens, with the area of the screen that matches the area of the square in front of the portal. They control the motion of a spotlight that outlines shadows of people who visited San Giovanni Evangelista during the previous day.

Sysetm
The system consists of a Kinect (used as a depth camera) and a projector, both located on the high flagpole. It covers all the space available and works in two states: day state and night state. During the day the Kinect detects people who pass by or stand in the installation’s range. At night the projector shows an interactive spotlight that users can move with their smartphones to reveal the recorded shadows.

The system operates with a computer that elaborates incoming data from the Kinect and, through a Wi-Fi network, allows a smartphones to move its spotlight, which is projected onto the pavement. The system diagram is shown below:

Prototype

Our prototype was designed to be complete in all its parts and offer an experience close to the real one. The area of San Giovanni Evangelista was simulated in a large room, darkened to recreate a night atmosphere. As shown below, a wooden pole about three meters high and held up by a tripod, simulated the high flagpole of San Giovanni Evangelista, A projector and Kinect were mounted on top. The projections were visible on a wooden floor, different from the square’s stone pavement but irregular enough to suggest the final effect.

With the structure in place and tested to be sure of the system operation, we asked our classmates to walk in the installation area in the room and recorded their movements with the Kinect. The recorded data is a 3D matrix that depicts people’s presence, eliminating the background and transforming the detected figures into a point cloud; it is later possible to elaborate the data into a visualisation with openFrameworks. We developed the 3D point cloud from the Kinect into a 2D shadow composed of moving lines and pixels. This processed data was then ready to be sent to the projector, and the connected smartphone can move the spotlight within the space.

For more details:
http://www.interaction-venice.net/iuav12-13lab2/projects/fourth-wall/