< Back

Metanoia

A group project about an individual experiencing ego death and travelling through his mind.

Liminal spaces

Liminal spaces are the gaps in between destinations

- transitional periods such as a hallway between 2 rooms or the gas station stop in the middle of a trip. These liminal spaces can often be nostalgic or unnerving. The reasons for these feelings varies upon the individuals experience with similar spaces and invoke powerful emotions based on said memories. We could use liminal spaces in our capstone combined with auditory studies to produce a powerful visual experience focused on evoking emotions.

Shot ideas

Pirates of the carribean type of shot where the main character trips into water reflecting the night sky, and when he passes through the water he comes out of the other side in space looking down at the earth

Mirror ocean - salar de uyuni

Liminal space pool

Foggy gas station/playground?

Preproduction

For semester 1 we divied up the modelling workload based on what we wanted to contribute to the capstone. We set dates for both filming as well as modelling of the environments and Meta Human.

We don't know if we can use the AUT facility to get a photogrammetry of our actor's face for the Meta Human at this stage so we have decided to do it ourselves using our phones and going about it manually.

Modelling

Earth

Going into Unreal Engine I want to play around with the UI as I haven't really used it. There's a tutorial on youtube i'm following on how to create a space scene so i'm just following that for now. Maybe I can add my own details when I get comfortable enough with the UI.

After hours of tinkering around and scratching my head at UE's UI I finally managed to get some results with the atmosphere.

I downloaded an 8K image of earth both during night and day, as well as the specular map, normal map, and clouds from this website: https://www.solarsystemscope.com/textures/.

After applying these materials to my untextured planet I created a new material in Unreal Engine and dragged my textures into the material to connect them within the material editor.

I set off to get the clouds working. I created a material for the clouds and set the blend mode to translucent. I also set up nodes to animate both the clouds and the earth to rotate slowly rather than just being static.

Then I duplicated the earth, created a new material for the star texture, resized the sphere to triple the size of the earth and made the material two sided and translucent so the light would shine through and still be visible.

For the finishing touches I added film grain just to ground it a bit more.

Pool Room #1

Starting the pool scene I modelled the main pool structure in Maya rather than Unreal Engine because personally modelling is easier for me in Maya.

After modelling out the base of the pool I UV unwrapped it to prepare it for the import into UE

After importing it into UE I started building on the level using the modelling tab using cubes as the walls and UV projecting a downloaded 8k pool tile texture onto the base model and the walls. Then I added a custom water plane so I could make pools on multiple altitudes.

I was juggling between the textures above and this one but eventually decided to go with the texture below. Something about the dirtier wetness makes it look more impactful.

Pool Room #2

Starting off the the planning we drew up a quick idea of the pool room. This room will be smaller than the other pool room.

Starting a new UE level I started blocking out the basic design for the level.

Following the plan I inserted edge loops to create the stairs down to the bottom segment.

After this I went ahead and added the pillars and applied the pool textures to everything. I used the projection mapping to make sure all of the squares were correctly lined up and sized.

Then I added some lights and adjusted the brightness to get the right ambience.

Meta Human Prototype

To start off my prototype I imported a pre made metahuman into a new level.

Then I went about setting up an IK rig for both the Mixamo animation maniquen and the Meta Human. I had to group the major bones together and then I retargetted both of the rigs together.

Then I had to go in and pose the Metahuman rig to match the t pose of the Mixamo rig and double check everything was linked together nicely.

Then all I had to do was import the Metahuman into my space scene and then link the Mixamo animation to the Metahuman model.

Reviewing the footage, I wasn't happy with it so I took the animation into maya and slowed it down drastically. I also straightened out his wrists so they arent pointing upwards and his spine so he's not hunched over so much. Then I dimmed down the stars and widened the camera to give more of a sense of scale. Then I repositioned the camera and the meta human to overlap as well as giving him an edge light for dramatic effect.

Filming

All of our filming for this project is gonna be infront of the greenscreen. To get the actions for the environment right we measured out the distances with coloured tape along the floor so our actor knew where to stop. Because of the nature of the motion capture for the camera and the limitations of only 1 wall being green we used a green cloth over a tv on a stand as well as over a broom to extend the green screen wherever we needed it to.

The footage we shot previously had corrupted so we had to re shoot every shot we had done. It only took us about 4 hours to refilm everything and because we knew how we wanted them they turned out better quality this time around.

Compositing

Bringing the greenscreen footage into nuke I played around with different tree's but ultimately decided that a basic keylight in the right spot worked the best.

I roto'd a rough outline of felix to get rid of the lights and walls that weren't taken care of by the key.

Custom Meta Human Creation

Over the break we a program called agisoft to create a photogrammetry of felix's face using amani's iphone. At first we tried using felix's old dlsr camera but the file conversion was messy so we opted to amani's iphone. At first we tried taking the photos outdoors but the harsh lighting made the shadows on felix's face too dark so we created an artificial lighting environment inside felix's room which resulted in flatter more consistent lighting.

After exporting the face mesh and textures out of agisoft I imported them into unreal and used the agisoft mesh to create a meta-human mesh. I tried activating every point and less points to see which would capture his likeness the best. The scan without the eyebrows, cheek lines and ears turned out to be more accurate so I went with that one.

Now I need to learn how to use the groom function in houdini to make custom hair and merge it onto the meta human.

I followed a tutorial on how to groom in houdini. I found that all of the tutorials were for short straight hair though so I had to play around with long wavy hair. It still needs a bit of refining but I managed to get the basic shape down.

I played around with the groom a bit more and managed to get it curlier by using the clump node and other groom nodes so that it is closer to the model's hair.

I imported the groom into UE5 just to see how it would look and there's some weird lighting issues with it as well as the hair rendering badly. I'll need to tinker around with it within houdini to get it to an acceptable standard.

I played around with the hair density in Houdini as well as restructured the guides just to see how it would look. Alot of this is just throwing it at a wall and seeing what sticks

It's getting closer and closer to how I want it. Though the hair colour is too light and it's a bit thin in some spots but overall I'm happy with the direction it's headed.

I tried importing the groom from Houdini to UE5 as an alembic file but when I tried to import it at the normal scale and tried to bind it to the metahuman it came up with an error. I looked up how to trouble shoot it and people were saying to upscale it by 100 which worked. But when I binded the upscaled groom it appears at his knee caps and I can't change the size of the groom after attaching it. I will have to wait next week to get help with this.

While i'm waiting for some help on the hair I decided to work on the salt planes scene because we were having some trouble. At first I used the water system in UE5 to see if I could make it more reflective but I couldn't figure it out. I found that flipping the water plane upside down made it really reflective but had this weird lighting problem on the edges so I looked into a different way to reflect the sky.

Also with the base lighting system in unreal I couldn't figure out how to keyframe the times of day to make a timelapse so I looked into a plugin called dynamic skies. This allowed me to set keyframes for the time of day, customize the clouds etc. I also looked into reflective planar's and ray tracing. Eventually I got it to work using a combination of the two but the angles I can shoot are very resticted.

We figured out that it was importing weird because the hair in houdini was too close to the ground. To fix this I imported the face mesh of the head into Houdini which would accurately show me how high to move the groom in order to import it right into UE. I used a transform node to move it up and match the height of the model and then exported. I had to play around with the scale as well but I eventually got it right and managed to attach the groom without any errors,

After importing it into UE I changed the colour of the groom through the hair material closer to Felix's hair colour. I also played around with the simulation settings to get the floating effect for the space scene.

Salt Planes

I found a sky asset which allowed me to keyframe the time of day for our timelapse called 'Ultra Dynamic Skies'. I also tinkered around with different water materials, as I felt that the reflective planar was a little unstable to use.

I played around with the different reflection types in the PostProcesVolume. I found that 'Lumen' was better at reflecting the objects closer to the ground, but less effective at reflecting the sky, losing detail. However 'Screen Space' worked oppositely, making really clear reflections of the sky but terrible reflections of the objects closer to the ground. I can use 'Lumen' for the shots with the metahuman and live footage close to the water. And then use 'Screen Space' for the timelapse footage where the scene transitions from day to night.

I also set up the live action footage with the help of Amani. We're playing the footage as a plane in the scene itself. So any lighting changes we make further down the road will immediately affect the footage which is nice. However the tracking wasn't perfect and was really jittery so I had to go in and manually keyframe 200 frames to line up the footage with the motion of the camera. I used the 3D grid on the floor as a reference point and manually matched is left foot to the same spot.

Then came the water simulation for the metahuman when he falls into the water. I found a falling animation off of mixamo and applied it to the metahuman. I also inported the animation with a skin into houdini and used a flip tank to simulate the splash. I tinkered around with the settings until I got a result I was happy with. After I cached the simulation I exported it as an alembic file and imported it into unreal engine. I attached the water material the simulation and placed it just under the plane of water. The simulation has these weird bumps that I couldn't flatten in houdini so I just lowered it under the plane.

Next came the water simulations for the live action footage. I tried using rokoko to automatically make an animation clip using ai so import into houdini, and then use the mesh of that to simulate the water. But it didn't track properly so I just used 2 tubes and imported the footage into houdini as a back drop. I manually animated the tubes forward in timing with his steps, and then tinkered around with the settings until I was happy with them. Then I exported them into unreal engine, put the water material on the geo cache and timed it to the animation of the footage in unreal engine.

After I lined the water simulation up with the footage I tinkered around with the lighting of the dynamic skies plugin. I will probably have to colour grade the original footage in nuke as well, because the base footage is still a little too washed out and needs some prior grading before looking fully intergrated.

For the scene where the character falls into the water, I made a quick bubble simulation in Houdini following a tutorial I found on YouTube. I used POP fluid and rendered it out using Mantra before compositing it in Nuke.

I also composited the transition between the eye and the space scene as a reflection on his eye.