COMPOSITION: MATTES, MASKS & BLENDS

 

Some examples in this chapter may not work correctly in QLab 4.2. All have been tested in 4.1.7 with OS 10.10.5 and OS 10.13.3. The workspaces using blend modes require Syphon Quartz Composer to be installed which can be downloaded from https://github.com/Syphon/Quartz-Composer/releases/tag/4

Three features which are not included in the QLab video features list are:

The ability to add more than one video effect to a video cue, in the way you can add multiple effects chained one after another in audio cues.

Blend Modes, which would allow full video compositing and  Photoshop style effects, with many layers and masks composited together to produce a final output.  

Cropping and masking  of images in video and camera cues.

This chapter examines ways in which QLab can achieve all the above and how these techniques can be combined to produce complex layered compositions.

All these techniques require a lot of computer power and you are not going to be able to composite many layers of HD video at full frame rate with blend modes and masks. But even using a single layer of video with a few layers of stills can still yield good results and extend the video capabilities of QLab in all sorts of useful ways.

Applying more  than one  video effect on a cue
This example examines how we can use 2 video effects simultaneously on a camera feed. We’ll use a FaceTime camera which first fades up blurry and with super-saturated color. Then, on a cue it fades into a Sharp Black and white image. Here it is in action

In a node based editor the set up would look something like this:

2effect chain

To achieve this in QLab we take advantage of the fact that any surfaces that are assigned a Syphon output appear as camera inputs in Camera Cues. (Syphon is a system for routing video between applications. If you are not familiar with it you can find out more here)

We create a Syphon surface that is the same size as our output and assign that surface as a camera input.

2effectssettings

(Currently QLab does not update the names immediately.  After creating and naming Syphon surfaces you will need to close,  and re-open the workspace)

Here’s the cue list:

2effectcues

In the first group cue the first camera cue has the camera assigned,  the color controls video effect applied in the Video Effects tab,  and outputs to the Syphon surface.

The second camera cue has the Syphon surface as its camera, has the blur effect applied and outputs to the main projector output.

You can download an example workspace here

Masking a camera input

For video cues we are able to create prerecorded content in the aspect we want to project it and generally clean things up so that anything that we don’t want to be included in the frame is edited out. With a camera input, we have no such luxury. Let’s say we have a 1080p  camera focussed on an actor with crew passing up and down in the left of the shot. We want to have a circular picture and want to mask out all the extraneous activity on the left-hand side of the camera.  We want to project this image in front of a background image on the projector and make the image recede (by making it smaller) over 30 secs on a cue.

If we just wanted to project a circular image by itself we could mask it using a black image with a circular transparent area (saved as a .png with alpha)

We could put the camera cue and the mask in a video cue in a start all children group and produce  this:

masked image in group

But when we try to put a background image behind  this we have a problem:

circlemask on background

If we just wanted to project a circular image at a fixed size on top of the background everything would be quite straightforward. We could create a surface which was always on top of the main surface with pixel dimensions of say 600×600 and have a circular mask on that surface.

The set up for the circle surface  would be:

circle surface

Note the dimensions of this surface, the layer and the image in the Mask well.

Masks for surfaces are black and white images where the black portions of the image mask and the white areas correspond to the areas of the image to be shown. Do not use masks with alpha channels.

The output, when the camera feed is projected using this surface on top of the main surface (set to the bottom layer), might look like this.

Circle surface screen

As long as the circle is always going to be in exactly that position relative to the screen, and the same size, this is a good method. But if we need to change the position or size of the camera feed, we can’t, as there are no cues or other methods that can control the set up of the surface.

If we need to resize the image or move it around then we need to find another way of doing this. We need to create a mask within the cue that outputs the camera feed with a proper alpha channel to the main projector surface so we can still see any images on layers behind it.  As we will then have the masks with the cue, we can scale the circular image, move it, change its opacity, in fact just treat it like a regular video cue.

So, we want to find a way to create a video effect, on the camera cue, that can give us a circular mask that we can control.

In order to create new filters for QLab video and camera cue effects we can use Quartz Composer, which is a  node based visual programming application for creating video effects and rendering them.  If you have never used Quartz Composer before there is a handy introduction on the Figure53 website. It was written for QLab 3 but almost everything is applicable to QLab 4 as well.

QLab 3 and Quartz Composer – Custom Image Filters

In Quartz Composer the patch for our circular mask looks like this:

Circular Mask QCPatch

At the top left and right are the Image (required)  in and out ports which give QLab a way of sending and receiving video to the effect. All QLab custom Video effects constructed in Quartz Composer have these. At the end of this tutorial is a link to a basic template for when you are ready to start experimenting with your own.

The node on the left-center called Image is the mask image. This contains a grayscale image where black is the area to be masked and white is the circular area we want the image to show in. The image is embedded in the filter so doesn’t need to access the original mask image.

Moving to the right the image is fed to the Affine Clamp node. This basically just allows the mask to be resized by scaling and moved around the main image to select the area of interest. It can also be rotated (but this isn’t very useful with a circular mask).

Each of the transform parameters like scale are published and are split to input ports (the little nodes on the left). The ports will appear as sliders in QLab to control the mask.

The image then passes to a Blur node which allows us to soften the edge of the mask and then is combined with the main image in the Blend with Mask node to achieve the masking. We then ensure that the output image is constrained to the input image size with the Image Dimensions node which feeds the information regarding the pixel size of the input to the Image Crop node. Finally, the image is fed to the output port where it will reappear in the QLab camera cue. The video effect will look like this in the video effect tab of the inspector.

video effect controls

These are good starting values for a 720p camera feed.  We use the origin, translate and scale controls to select the area of interest in the camera image. don’t worry where it is on the screen at this stage Here is what the cue looks like in QLab.

QLab Qc cue

We can now use the controls in the Display and Geometry tab of the inspector to position and resize the masked image to place it where desired on the background.

QC geometry

Finally, we can add some fade cues to make the circular image recede and fade. We use separate fades for the opacity and scale so they can use different curves.

recede cues

The sequence then looks like this:

You can download this workspace here

The following advanced composition techniques require Syphon for Quartz Composer version 4 to be installed on your machine which can be downloaded from

https://github.com/Syphon/Quartz-Composer/releases/tag/4

Blend Modes

Before we combine all the techniques the final method to explore is blend modes.

In this example, we will composite some ripped graph paper over a background using an overlay blend

Here are the 2 images

Surface and cameras settings:

Screen Shot 2017-11-18 at 02.21.23

All the surfaces have the same dimensions.

There is a conventional surface called Projector with the projector screen assigned

There are 2 Syphon surfaces:

Main Image which is the surface the image is composited on.

overLay Image which is a Syphon surface which a Syphon client within the Quartz Composer patch, which does the blend, will use

And here are the cues:

Background sent to Main Image Syphon Surface.
(It also has color controls selected in the Video Effects tab for fine tuning)

Screen Shot 2017-11-18 at 02.17.53

Graph Paper sent to  overLay Image Syphon surface.
(Again, color controls in  Video Effects )

Screen Shot 2017-11-18 at 02.18.00

Camera Cue with Main Syphon surface fed to Projector with add mode Quartz composer custom composition in video effect tab.

Screen Shot 2017-11-18 at 02.18.03

The Quartz composer patch is fairly straightforward and has no controls:

Screen Shot 2017-11-18 at 02.37.32

The Syphon Client in the top left is set up to receive the image from the overLay Image Syphon surface. i.e the graph paper

Screen Shot 2017-11-18 at 02.41.28

The image port below it receives the video from the camera cue the video effect is in.

The Overlay Blend Mode  node uses the syphon client as  the image source and the image from the cue as the background and after blending sends it to the output image node.

The node either multiplies or screens the source image samples with the background image samples, depending on the background color. The result is to overlay the existing image samples while preserving the highlights and shadows of the background. The background color mixes with the source image to reflect the lightness or darkness of the background.

Here is the Projector output:

Screen Shot 2017-11-18 at 02.26.31 (2)

You can download the workspace here

A Complex Composition

We now have all the methods we need to create a composition involving a background, a masked camera cue, and a foreground overlay with the final result given a soft-edged border which will look like this:

compositing

Here are the video settings for the workspace:

comp settings

We have a main video surface which has a mask applied on the surface, and 4 Syphon surfaces (the same pixel dimensions as the main surface) which we will use to route video through to achieve the composition.

Our source camera is in the camera patch at position 1. The 4 Syphon surfaces are in positions 2 -5.

Here are the cues

The background is output to the Main Image Syphon surface

Screen Shot 2017-11-18 at 13.37.03

It has color controls applied in the video effects tab for fine tuning.

Screen Shot 2017-11-18 at 13.37.12

The camera feed has the camera you want to use selected and is sent to the color correct Syphon surface.  It  also has color controls assigned in the video effects tab

Screen Shot 2017-11-18 at 13.37.22

The Mask Camera cue uses the color correct Syphon surface as its camera (This has the color corrected camera video on it) and outputs to the Main Image Syphon surface. The custom geometry in the tab places the masked video scaled and positioned on the output screen.

Screen Shot 2017-11-18 at 13.37.36

It has the circle mask custom composition applied in the video tab, with the controls to select the area of interest within the camera feed and to remove the unwanted part of the picture using a circular mask.

Screen Shot 2017-11-18 at 13.37.42

Here’s a reminder of the Quartz Composer patch for the mask

Circular Mask QCPatch

The graph paper image is in a video cue and is sent to the overlayImage Syphon surface. It also has color controls applied for fine tuning

Screen Shot 2017-11-18 at 13.37.46

The Blending Camera Cue takes the Main Image Syphon surface as its input, and outputs to the Final Image Syphon surface.

Screen Shot 2017-11-18 at 13.37.57

It has the add mode custom composition applied in the video effects tab

Screen Shot 2017-11-18 at 13.37.54

This composition picks up the overlayImage Syphon surface (which has the graph paper image on it from the Image to Blend Over cue.) and overlays it on the camera feed for this cue (which currently has the Background and main camera images pre composited on it)

Screen Shot 2017-11-18 at 02.37.32

Finally, in the Composition output cue the Final Image correction Syphon surface is used as input and is output to the Main surface with final image color correction applied in the video effects tab.

Screen Shot 2017-11-18 at 13.38.00Screen Shot 2017-11-18 at 13.38.04

The Main Video Surface has a mask applied to give the whole composition a soft edge (This is a black and white mask).

MASK

finalmask

You can download the Workspace here

compositing

Basic Template Cue

Here is a link to a basic Quartz composer patch that is a good starting point for rolling your own effects. It has the required input and output ports and the dimension and crop nodes that will ensure the output is as expected. Download here

The basic patch looks like this.

QCbasic patch

Here’s an example of how you might edit it to provide a simple filter

In Quartz Composer click the Patch Creator button

In the Patch creator window that opens your main area of interest will be patches of the type Image/Filter. In this example, we will use Edge Work. Drag the edge work patch to the editor window  and connect it between the image input port and the crop node like this

edgework

Edge Work has one control called radius. We will want to include this as a slider in the QLab video effect tab when the effect is used as a custom composition.

To do this right click on the Edge Work node and publish inputs/radius. right click again and insert input splitter/radius. This will create a node called radius.

Open the patch inspector in the editor window, click on the radius node and in the inspector, select settings on the pop-up menu.

Input the range you want for the slider. For this filter, a min of 1 and a max of 100 will work well.

min max

Save as Edge Work.qtz. It should now look like this.

edgeworkcomp

You can now use this as a custom composition in the video effects tab of any QLab video or camera cue.

If you edit the patch, e.g to change the max value to 200, save the patch after making the change. To force QLab into using the new version go to the Displays and Geometry tab in the cue and click on the video surface selector (without changing its value). The effect should update.

A Finished QC Set

As the grand finale of this chapter, we’ll now distill all the techniques above into a workspace that uses only 2 custom Quartz Compositions.

The first allows 1 of 6  mask shapes to be selected, including a mask that can receive input from any video (or camera cue) in the workspace.

The second allows 1 of 21 different blend modes to be selected.

Mask Selector QC

The Mask selector  QC is an improved version of the circle mask we used earlier.

To make it more useful we are going to add separate controls for x scale and y scale of the mask and add a master scale so that the overall scale can be adjusted while retaining the x/y scale ratio.

scaling

There are 2 maths expression nodes which are set to a*b. This allows the value that the x and y input splitters receive from QLab to be multiplied by the Master Scale value.

In the circle mask QC we only had 1 mask image set to have an embedded grayscale circular mask.

circlemask

We’ll expand this to 5 different shapes and provide a menu control in the video effects tab of QLab to select which shape to use.

When given blur and rotation controls, these   provide  a good selection of mask shapes

In addition, there is a Syphon input that will use anything on a QLab surface that is named syphonMaskFeed as a mask. This can be a grayscale still or even a moving image so animated masks can be achieved.

In the Quartz Composition, we provide the means of selecting which mask to use, using a multiplexer node.

multiplexer

Each of out mask image nodes is connected to a multiplexer port and the source index parameter is published as ‘shape’ and its input split to make it available to QLab in the form of a menu.

We set the names we are going to use in this menu in the node settings of the shape input splitter.

indices

This appears with all the other input splitter node names  in the controls of the video effects tab in QLab for any cue that has this QC custom composition applied.

shape menu

Each splitter node is numbered in Quartz Composer as part of their names in order to force the controls to display in a sensible order instead of alphabetically.

Blend Mode Selector QC

A similar method is used with the blend mode selector which is based on the add mode.qtz composition we used earlier.  We use a couple of multiplexers with 2 inputs to allow the foreground and background images in the blends to be swapped and connect the outputs of these to every blend mode available in the standard patch set.

We then use a multiplexer with 21 sources and connect the output of each of the blend mode nodes to 1 of the source inputs.

blendmonster

The source index is published and it input split and all the indices named as before in the settings. In QLab the blend controls in the video effects tab of any cue that has this composition applied looks like this:

blend controls

end menu

In the example QLab workspace provided below, the following surfaces and camera patches are used.

examplevidset

Here’s the cue list

exworkspace234

The stone image is sent to the composition Syphon surface as the background

A video cue of driving past flats at night is sent to the overlayImage Syphon surface which feeds the Syphon node in the Blend Selector Quartz composer patch which is used in the camera cue below..

A Camera Cue has the composition surface assigned as its camera (which currently has the stone image on it)  and has the blend selector QC  applied as a custom composition in its Video effects Tab. This cue outputs to the surface named finalColorCorrect

A video cue with a space image still is sent to the same surface but is masked using another instance of the mask selector composition applied in its video effects.

Finally, a camera cue uses the finalColorCorrect surface as its input, has some color controls applied in its video Effects tab and outputs to the Projector Output surface.

This looks something like this:

Screen Shot 2017-11-20 at 17.39.16 (2)

You can download a file which contains the workspace, the 2 Quartz Composer patches, and the footage used in the example here

Chapter Author: Mic Pool

Chapter Graphic remixed  from original photographs
including Marble Wall © HaleiLaihaweadu
Licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Redistributed under the same license.
Creative Commons License

MenuGraphic