Pose Animator p2p
20 Oct 2020 - Jake Sherwood
Pose Animator w/ Friends
Pose Animator - Dancing Together
For my final for The Body Everywhere and Here I worked further on the peer 2 peer pose animator.
Using PoseNet and the Pose Animator lib I was able to create a fun collaborative space where you can bring illustrations to life and dance together.
The things I still needed to work on from last week were:
1) Figure out how to run note on localhost only. I’m working on Lisa’s suggestion of using glitch.me
2) Update drag and drop to be able to change both illustrations by dragging onto poseNet windows.
3) Show/hide poseNet windows
4) Add additional interaction affects?
5) Additional simplified illustrations
6) Refactor duplicate code.
I accomplished most of those but still need to clean up the code more and sort out a few last bugs.
As in the last post the biggest hurdle here was just finding out what did what in the massive Pose Animator library. It was definitely daunting but I learned a lot in the process.
To get it up and running with a remote client, I tried a number of things. I tried glitch, I tried running it on my on server, setting up a proxy and I tried just exposing the ngrok address.
All of which had their own problems.
Eventually with a little help, I was able to get it working on my own server.
I ended up having to build all the pose animator files on the server. Then I set up a simple expressjs app to run a server with the built files, which were pointed to my ngrok address exposing my local peer.
Once I had the peer connection sorted, this took a few days, I was back to developing code.
The original drag and drop functionality allowed for dragging SVG files to the canvas window to change the animation svg. But I wanted to be able to control/change both local and peer illustrations.
I didn’t get all the way through the code clean up so there is some dupe code here, but to make it work I ended up making a couple new functions to drag and drop and then parse local and peer SVGs. That allowed me to drag to my local poseNet canvas window or the peer window.
Next was hiding the poseNet data windows. This was relatively easy but I probably spent more time than I should have on the button. Clicking the button just shows and hides the poseNet data windows container element.
I also wanted to add some more playful interaction between the the 2 users. I ended up with a somewhat buggy way of changing the SVG by both users touching right hands (wrists). It was working when I ran it locally but with a remote peer it seemed much more buggy. But when it worked, it would cycle through an array of available SVGs.
I also worked with interactions of just swapping the 2 user’s SVGs. But allowing them to try out multiple illustrations if they didn’t have illustrations of their own seemed fun.
I just reverted back to the pose wrists touching interaction. This way the remote peer can touch wrists and change the illustrations for the other user.
For some reason this work more reliably.
I’d like to add functionality to send the SVG used to the peer, but thought it might be problematic if either peer was using custom SVGs that weren’t in the repo.
I did get a couple of new simplified illustrations done, but they still had strange stretching problems.
These worked better than my more complicated previous illustrations, but some more fine tuning is required to get them to look good when animated.
I also cleaned up the code a bit but have a bunch more to do.
I was still getting some strange peer pose data but it only appeared to really happen over a remote connection. Not when I was connected locally.
I also want to create a good README.md file for the GitHub repo incase others want to try it out.
All in all I’m happy with the result. I do plan to put a little more work into it. As Fernando mentioned I can see this being a really great storytelling tool and I want to explore that direction.