2 Failures and Lessons - Video Installation
06 May 2021 - Jake Sherwood
2 Failures and Lessons - Video Installation
The final project for 3d three ways was to take what we’ve learned over the course of the semester and apply it to a video installation piece.
I tried a couple of different approaches, and while neither of these worked out to the degree of fidelity I had hoped, I experimented a lot and learned several things.
Lesson #1 / Failure #1 - Projection mapping
Originally my idea was for a projection mapping piece. My thesis focuses on climate change and consumption and used styleGAN latent spacewalks as a story-telling medium. Wanting to explore that idea further, my concept was an immersive experience projection mapping a hallway with the styleGANs.
I had a small pico project that I was hoping would cover the small hallway in my apartment… as Ziyang expected, it did not work.
I experimented with madmapper and set up a series of scenes to incorporate the idea of the damage humanity has done to the planet.
Using styleGAN models trained on smog, factories, floods, fires, droughts, and waste, the concept is these generative images are our potential futurescapes.
I also worked on a new animation of a world coming into view and then shattering. I used this model by Akshat from sketchfab. Conceptually this signifies the destruction of the earth.
Experimenting with the shatter and bullet Maya fx, I created the effect of the earth falling apart. This tutorial on the bullet effect helped me get it set up. I had difficulty rendering this out but was able to render it via playblast, which worked for my purpose.
I used the great collab audio track between Kim Anno and ClimateMusic - This is what Climate Change Sounds Like.
I see this as a prototype for a more well-executed experience I hope to create in the future.
mapped still
Failed hallway map
Wall map with scenes.
Earth Shatter asset
So while this didn’t work out as I had hoped, I was able to learn and experiment with MadMapper and projection mapping. Something I hope to explore further in the future.
Lesson #2 / Failure #2 - LED screens
We also looked at works with LED screens, and I just happen to have some dot-matrix LED screens I got for another project but did not use.
Since my projection mapping idea didn’t fully workout, I would try to pull something together with a LED screen piece.
Having recently trained a new styleGAN model on a data set of lava, the concept for this piece was to create a lava floor or a pile of lava. Also, a climate change piece, conceptually speaking to a warming planet.
I used a Raspberry Pi to drive the project and an Adafruit dot matrix hat and the rpi-rgb library by Hzeller. I also experimented with the Adafruit rpi-fb-matrix this lib is supposed to allow you to play videos on the dot matrix panels. Adafruit, however says it will not work on a Pi 4, which I confirmed.
With the help from a buddy who had worked with these screens prior, I learned I could just run the stylaGAN animations as a gif, and things started working better.
I still had several hurdles. Part of the config for running the scripts with the dot-matrix libs defines the number of pixels and chained panels. It was a challenge to get it set up correctly. Issues with sizing and how many pixels to set in the config took some trial and error to configure properly.
Eventually, I had to resize my original gif to be the number of total number of pixels I had on my LED panels, 512 x 32. Once I had the content sized appropriately, things work out easier.
Still, there were power issues. I only had one 5V power supply, and it only puts out 2A, which isn’t quite enough power. I have other higher current supplies, but the matrix hat has a limiter and won’t work with anything over 5V.
With my 5V 1000 2A running, the animation with close to full brightness causes the hat to short and lose power. To work around this, I lowered the brightness to 45%, which was still plenty bright, and reduced the current draw enough to prevent shorting. An unfortunate byproduct was more glitchy playback.
Also, the GAN is almost incomprehensible, resized, and played back in this low fidelity way. I don’t think anyone would realize it was a styleGAN unless they were told. Higher-resolution panels may make this less of an issue.
LED pile
LED dot matrix panels
Lava GAN
LED short
Lava floor Video
With both of these experiments, I had problems with refresh rates and documentation. I need to do more research into how to handle that.
Lessons:
1) Spec out spatial requirements better and use proper hardware. e.g., better and multiple projectors
2) Allow time for rendering issues
3) Select proper power supply for LED matrices
4) Think about content and whether it will be successfully rendered on a dot matrix
5) TODO - figure out refresh rate documentation.
Overall I’m happy with the experimentation done and plan to explore both these mediums further.