3D Land - Liveweb Final

11 Dec 2020 - Jake Sherwood

3D Land - Liveweb Final

Liveweb Final Idea

3dland is a video chat experience set in a 3d spacescape. Using Threejs, Peer to Peer connections w/ simplepeer.js, users are able to text chat, video chat, and naviate the 3d space.

For my LiveWeb final I continued my work with webrtc and simplepeer, and brought it into a 3D world.

Following this turtorial I modeled my first low poly 3d planet.

Which I used as the stage for 3D land.

Most of this project was spent getting to know threejs. It’s a massive library and as this was my first time working with it, I spent a lot of time in with the documentation.

My original plan was to use Aidan’s webrtc_threejs template. But after I got my space island environment working the way I wanted, moving everything to Aidan’s template proved problematic.

I ended up just going back to my previous code from tvland3 and porting it into my 3d land code.

This ended up working pretty well and allowed me to utilize some of the filter and chat code from tvland.

Some of the biggest challenges were “attempting” to understand quaternions, getting the peers to turn to face in the direction of movement, and adjusting / “tattooing” local and peer mesh textures.

After talking with Aidan, quarternions could be used to help with having the peers face in the direction of movement, but for some reason I could never get quite get it sorted out.

I ended up just sending a value to the peer and if it was X value then I set my lookAt value accordingly.

// Update when one of the users moves in space
    socket.on('peerPositions', posData => {
        let delta = clock.getDelta(); // seconds.
        let tempPosData = posData.pos.toString();

        let tempPos = tempPosData.split(',');
        for (let i = 0; i < simplepeers.length; i++) {
            if (simplepeers[i].socket_id == posData.id) {
                if (simplepeers[i].peerGroup) {

                    simplepeers[i].peerGroup.position.set(tempPos[0], tempPos[1], tempPos[2]);

                    let tempLookAt = simplepeers[i].peerGroup.position.clone();

                    if (posData.pos[1][0] === -100) simplepeers[i].peerGroup.lookAt(-100, 0, 0);
                    if (posData.pos[1][0] === 100) simplepeers[i].peerGroup.lookAt(100, 0, 0);
                    if (posData.pos[1][2] === -100) simplepeers[i].peerGroup.lookAt(0, 0, -100);
                    if (posData.pos[1][2] === 100) simplepeers[i].peerGroup.lookAt(0, 0, 100);


                }
            }
        }

    });

Locally you can rotate around the X & Y axis and eventually I would like to get that worked out with quaternions so I can send it to the other peers.

Getting the peer textures to update took a bit to figure out.

I was able to see my changed textures in the terminal but for some reason I couldn’t get them to update.

I finally found this stackoverflow which showed that I needed to traverse the mesh object and reapply the texture.

Once I had that I was able to update the textures locally and for the peers.

function updateCubeMaterials(mat) {
    redCube.traverse(function (child) {
        if (child instanceof THREE.Mesh) {
            child.material = mat;
        }
        redCube.geometry.uvsNeedUpdate = true;
        redCube.needsUpdate = true;
    });
}

Also to get animated gifs to work on canvas DOM elements I used supergif.

Supergif allowed me to draw the gifs to canvases off screen and then use those to create new three.js materials.

The idea of applying gifs or tattoos to the peer participants started as something funny, but after feedback from the class I began think further about what this could mean.

Do participants like being “gif’d”? Does it feel like a game to try to get close to someone to “gif” them?

Ultimately this project is meant to give people a more engaging way to have a video chat experience and break us out from the linear boxed zoom world we’re spending soe much time in.

Customizing yourself and those around you add to the experience and allow each user to tailor it to their own liking.

The gifs can be seen as gifts and / or flare given to the people you are chatting with.

In future versions I could see gaining gifs being some sort of status symbols. Perhaps certain gifs could unlock hidden features of the world.

Overall I’m very happy with how this turned out. I learned a ton about 3d and accomplished most of what I set out to do.

I plan to take this a bit further and allow for multiple terrains / environments and allow for users to drag in their own gif content.

Github code here.