Sound Visualisation Testing

Now that the room is coming together and most of the rotoscoping is done, it's time to start experimenting with visualising the music like I did in the animatic. Here are a few things I came up with.

White and Black

Black and White

Strobe

On this one I decided to try alternating between black and white to see what it looked like. It creates an interesting strobe kind of effect that could possibly help to make some parts stand out.

3D room and objects

After taking measurements of the objects, I made templates for each of them in Illustrator and began trying to assemble them in After Effects.

When I made the first templates, I was converting Feet and Inches to pixels, which didn't work out accurately because of there being 12" to 1ft instead of 10. Either way the first one wasn't too far off in terms of proportions. I managed to comp some of the rotoscope footage I've completed so far into this test to have a rough go at the beginning of the animation. 

After finishing this test I decided to have a go at reworking the measurements on the templates. This time I converted them from Feet & Inches to centimeters and adjusted the templates to have 1 pixel for each centimeter, this gave more accurate proportions. Once all templates are finalized I will scale up the artwork to around about double the size needed so that when I import them into AE and scale down by half, it will create a super-sampling effect meaning minimal pixelation! The downside to this is that using such high resolution images can be very demanding in terms of computing resources but I shouldn't have any problems there.

Here's an example of the templates I created in order to make the 3D room:

More Measurements

Here are some measurements I took of the objects I want to include in my After Effects room to aid the building process of their digital counterparts. I did this mainly to ensure the proportions are roughly correct so that the rotoscope animation roughly matches the perspective of the room and therefore works well when composited. 

Room Development

The room in my animation is to be based on my own room so I started with a few drawings to plan the layout of the room.

Next I will photograph and measure each object I intend to include in the After Effects room.

• Speakers
• Amplifier
• Drawers
• Doors
• Desk
• Monitors
• Window
• Noticeboard

After Effects Room Test

Now that I'm confident I can create the rotoscope sections I thought I better do some experiments in After Effects to make sure I can create the 3D room.

To save time I created the surfaces in Illustrator, keeping everything flat for now just to keep things simple. I layed it out like a box net:

Firstly I tried importing the Illustrator layers directly into After Effects but for some reason the layers don't display in the renders. At first I thought it was an issue with the 3D rendering engine in AE but after some testing I noticed it was only affecting the Illustrator layers. 

I then tried exporting each layer separately as jpegs and tried making the room again, this time it rendered fine.

One noteable thing about this method is that high resolution (at least 4k) images are needed to avoid pixelation.

The following video is a spin showing the basic set up with two lights.

It works mostly as I imagined. Once I create the individual objects to go within the 'room' I think it will look much better. There will be more depth and the objects will also cast shadows making it all look much more convincing.

I then imported my rotoscope test into the composition and aligned it within the view of the camera, I'm pretty happy with the results of this:

When I create the final room with proper proportions, alignment, colour and possibly texture, the combination of these two techniques will look quite visually pleasing.

Rotoscope Line Test

So now that I know what I'm doing and how it's time to start some technical experiments/practices.

I recorded some footage from a first person perspective and I have to say this was harder than I imagined. It was very difficult to get the motion I wanted with the correct timing for the song.

After a few takes I went through them and trimmed down a take that I thought would work. One thing I have to note with this first attempt is that the footage I recorded is awfully noisy, this could simply be down to low light in my room in which case I'll redo it when there is daylight available through my window. Other causes could be the 10-20mm lens I was using or perhaps the settings. I will attempt more footage tomorrow during the brightest point and I will also use the kit lens to see if that's any better.

Here's what I came up with from this footage:

To composite this animation with the 3D room in After Effects, I will need to export with the Alpha channel selected meaning I need to block the colour in for the hand. In order to do this I need to make sure I'm connecting the lines whilst drawing out the frames because it's a huge hassle to go back and fill in the gaps on each frame so that the paint bucket tool can be used. I think that this actually looks quite good as it is so I think this is pretty close to how the rotoscoping will look in the final animation.

Proposed treatment

As I've been developing these ideas I've had a general idea of the treatment I want to use but I now need to plan how I'm going to do each section.

The Room:

First of all, to create the room in which this animation is based I'm going to draw out the surfaces of the objects I want to include. I'll then scan and import these into After Effects, laying the surfaces out as 3D layers so I can essentially create a 3D room from 2D images. I feel that this will lessen the workload needed to create the background and allow for some interesting camera movements. (Mainly the pan around the speaker). 

Another method I considered for this was to create a panoramic image of the room and use the environment feature in Adobe CS6 but this would limit the camera to one stationary position within the scene.

Here is the tutorial I watched whilst considering it:

http://tv.adobe.com/watch/learn-after-effects-cs6/3d-environment-map/

The Characters:

As I've said in other posts I've been feeling quite inspired by the rotoscoping technique, particularly the style in Waking Life and A Scanner Darkly. My plan is to record video references from the various angle shown in the animatic so I can use this footage to animate the hand for the cough, the arm for the finger pointing, the close up of guitar playing, character standing up and walking to the window and finally the dialogue "Hey man it's raining". 

This animation will be exported from Flash using the Alpha channel to allow the animation to be composited in After Effects with my 3D room. I think that this will create quite an interesting effect that bares some reference to the Waking Life style.

The idea is that using first person perspective and rotoscoped arms will create an immersive effect on the viewer making them feel as though they're watching from within the animation almost as if it's their own experience/dream.

Sound Visualisation:

I plan to do the sound visualisation in Flash in a similar way to how I did it in the animatic. I will most likely create an alpha map from this animation and use it to animate the colours within the shapes. I would rather paint each frame but it would be far too time consuming.

Now it's time to start doing some tests!

Animatic

I decided to create my animatic in Flash using a graphics tablet as I find it's too difficult to plan out when drawing on paper. Here I was able to plan out my camera shots, character movements and audio visualisations.

Photo Animatic

To help figure out how I wanted the start of my animation to look I created an animatic using photos.

This will be really helpful as a reference for creating my animatic.

Rainy Day Storyboard

I've decided to go with the 'Rainy Day' idea because it's one I keep going back to and it inspires me much more than the others songs/ideas I proposed.

I drew up a rough storyboard of how I imagine this to play through but some aspects may have to be changed slightly depending on how well my planned treatment/process works for this animation. I've planned for the first 40 seconds (up to "Hey man it's raining) as a minimum length but I'd like to complete more if I can. 

Animating shapes and colours to music

Figaro the music with motion colors and shapes from Tonelf on Vimeo.

This animation looks to have been drawn frame by frame in with 2D animation software and I think this is a great way to create this type of animation but it can be extremely time consuming.

Another method of animating to sound/music would be to create the graphics in CAD software such as Adobe Illustrator and then mask these graphics in Adobe After Effects to create the impression of movement.

A group that have recently made use of music-syncopated animation for their music videos is the Arctic Monkeys. This video begins using a line like you'd see on a heart rate monitor, it eventually develops into other shapes that morph into each other. This work reminds me of Anthony Schepperd's animation style except much simpler. 

Arctic Monkeys - Do I Wanna Know?

Anthony F Schepperd

I recently came across the work of Anthony F Schepperd and have found his style to be quite inspiring. His work is detached from the rigid confines of reality but retains enough just enough reality to maintain an interesting balance. Much of the animation is in time with the music which gives the visuals an immersive and powerful impact. The vibrant colours, shape morphing, distortion and bizarre visuals conjure thoughts of psychedelic/hallucinogenic experiences.

This particular animation was created for Blockhead for their song The Music Scene. This shows there's a commercial viability for developmental animation such as this.

Inspiration

I began looking for inspiration for the visualisation of sounds in my animation and came across this animation.

The sequence from 2:00 - 2:15 is almost exactly what I was imagining for my animation. The use of vibrant colours and fluid shapes against a black background moving in time to the music works very well to create an interesting dynamic visual experience. I think that this could be used very effectively in my animation.

More inspriation came to me from Disney's Ratatouille. The film touches on the concept of Synaethesia with the visualisation of taste. Here is a compilation of the artist's work for this movie:

Here's how these animated vignette's were incorporated into the film. I really like how the background fades to black creating a sense of inner thought as though you're now inside the character's head:

Rotoshop - A Scanner Darkly

After watching Waking Life I felt quite disturbed and provoked in a strange way and it's not a feeling you normally get when you watch an animation. I think one of the main reasons it had the impact it did is down to the way it was animated.

I looked into this and found that the animation was directed by Bob Sabiston. He created a piece of software called Rotoshop specifically for this film. This software makes use of interpolation during the rotoscope process meaning the software will add the in-betweens, this greatly reduces the amount of time it takes to rotoscope the footage.

http://www.flatblackfilms.com/Flat_Black_Films/Rotoshop.html

"Rotoshop is the name for our proprietary rotoscoping software. Despite some appearances to the contrary, this software does not use filters, image-processing or any kind of motion capture technology. Rather, it is an advanced application for hand-tracing over frames of video. The program will interpolate between brushstrokes to save time and smooth motion, but the process is user-driven and can be extremely time consuming.

At this time, Rotoshop is not available outside of the company, and currently there aren’t any plans to market it. The software was developed in order for us to make animated films, and currently that is still the plan."

I really enjoy the style of animation created with the use of this sofware so I looked for other films created by Richard Linklater and found A Scanner Darkly. I found this to be fascinatingly put together and enjoyed it thoroughly. Unfortunately the Rotoshop software isn't available outside of the company but the techniques it employs can be applied within other software packages. 

"A Scanner Darkly is a 2006 American animated science fiction thriller film directed by Richard Linklater based on the novel of the same name by Philip K. Dick. The film tells the story of identity and deception in a near-future dystopia constantly under intrusive high-technology police surveillance in the midst of a drug addiction epidemic. The film was shot digitally and then animated using interpolated rotoscope, an animation technique in which animators trace over footage, frame by frame, for use in live-action and animated films, over the original footage, giving it its distinctive animated look."

http://en.wikipedia.org/wiki/A_Scanner_Darkly_(film)

Concept development

Here are a few quick sketches exploring some ideas for the first few shots of the animation. I want to make use of first person point of view and fluid tracking shots in order to create an immersive and dream-like experience.

More thumbnails

Here are a few more thumbnails exploring some of my initial ideas for the very beginning of the song.