WARNING: The underlying technologies used in this chapter are currently not functional in High Sierra or fully functional with QLab 4.2. They work in OS 10.10.5 with QLab 4.1.7.
Three features which are not included in the QLab video features list are:
The ability to add more than one video effect to a video cue, in the way you can add multiple effects chained one after another in audio cues.
Blend Modes, which would allow full video compositing and Photoshop style effects, with many layers and masks composited together to produce a final output.
Cropping and masking of images in video and camera cues.
This chapter examines ways in which QLab can achieve all the above and how these techniques can be combined to produce complex layered compositions.
All these techniques require a lot of computer power and you are not going to be able to composite many layers of HD video at full frame rate with blend modes and masks. But even using a single layer of video with a few layers of stills can still yield good results and extend the video capabilities of QLab in all sorts of useful ways.
Applying more than one video effect on a cue
This example examines how we can use 2 video effects simultaneously on a camera feed. We’ll use a FaceTime camera which first fades up blurry and with super saturated color. Then, on a cue it fades into a Sharp Black and white image. Here it is in action
In a node based editor the set up would look something like this:
To achieve this in QLab we take advantage of the fact that any surfaces that are assigned a Syphon output appear as camera inputs in Camera Cues. (Syphon is a system for routing video between applications. If you are not familiar with it you can find out more here)
We create a Syphon surface that is the same size as our output, and assign that surface as a camera input.
(Currently QLab does not update the names immediately. After creating and naming syphon surfaces you will need to close, , and re-open the workspace)
Here’s the cue list:
In the first group cue the first camera cue has the camera assigned, the color controls video effect applied in the Video Effects tab, and outputs to the syphon surface.
The second camera cue has the syphon surface as its camera, has the blur effect applied and outputs to the main projector output.
You can download an example workspace here
Masking a camera input
For video cues we are able to create prerecorded content in the aspect we want to project it and generally clean things up, so that anything that we don’t want included in the frame is edited out. With a camera input we have no such luxury. Lets say we have a 1080p camera focussed on an actor with crew passing up and down in the left of the shot. We want to have a circular picture and want to mask out all the extraneous activity on the left hand side of the camera. We want to project this image in front of a background image on the projector and make the image recede (by making it smaller) over 30 secs on a cue.
If we just wanted to project a circular image by itself we could mask it using a black image with a circular transparent area (saved as a .png with alpha)
We could put the camera cue and the mask in a video cue in a start all children group and produce this:
But when we try to put a background image behind this we have a problem:
If we just wanted to project a circular image at a fixed size on top of the background everything would be quite straightforward. We could create a surface which was always on top of the main surface with pixel dimensions of say 600×600 and have a circular mask on that surface.
The set up for the circle surface would be:
Note the dimensions of this surface, the layer and the image in the Mask well.
Masks for surfaces are black and white images where the black portions of the image mask and the white areas correspond to the areas of the image to be shown. Do not use masks with alpha channels.
The output when the camera feed is projected using this surface on top of the main surface (set to the bottom layer) might look like this.
As long as the circle is always going to be in exactly that position relative to the screen, and the same size, this is a good method. But if we need to change the position or size of the camera feed we can’t, as there are no cues or other methods that can control the set up of the surface.
If we need to resize the image or move it around then we need to find another way of doing this. We need to create a mask within the cue that outputs the camera feed with a proper alpha channel to the main projector surface so we can still see any images on layers behind it. As we will then have the masks with the cue then we can scale the circular image, move it, change it’s opacity, in fact just treat it like a regular video cue.
So, we want to find a way to create a video effect, on the camera cue, that can give us a circular mask that we can control.
In order to create new filters for QLab video and camera cue effects we can use Quartz Composer, which is a node based visual programming application for creating video effects and rendering them. If you have never used Quartz Composer before there is a handy introduction on the Figure53 website. It was written for QLab 3 but almost everything is applicable to QLab 4 as well. QLab 3 and Quartz Composer – Custom Image Filters
In Quartz composer the patch for our circular mask looks like this:
At the top left and right are the Image (required) in and out ports which give QLab a way of sending and receiving video to the effect. All QLab custom Video effects constructed in Quartz composer have these. At the end of this tutorial is a link to a basic template for when you are ready to start experimenting with your own.
The node on the left centre called image is the mask image. This contains a greyscale image where black is the area to be masked and white is the circular area we want the image to show in. The image is embedded in the filter so doensn’t need to access the original mask image.
Moving to the right the image is fed to the Affine clamp node. This basically just allows the mask to be resized by scaling and moved around the main image to select the area of interest. It can also be rotated (but this isn’t very useful with a circular mask).
Each of the transform parameters like scale are published and are split to input ports (the little nodes on the left). The ports will appear as sliders in QLab to control the mask.
The image then passes to a Blur Node which allows us to soften the edge of the mask and then is combined with the main image in the blend with mask node to achieve the masking. We then ensure that the output image is constrained to the input image size with the image dimensions node which feeds the information regarding the pixel size of the input to the image crop node. Finally the image is fed to the output report where it will reappear in the QLab camera cue. The video effect will look like this in the video effect tab of the inspector.
These are good starting values for a 720p camera feed. We use the origin, translate and scale controls to select the area of interest in the camera image. don’t worry where it is on the screen at this stage Here is what the cue looks like in QLab
We can now use the controls in the Display and Geometry tab of the inspector to position and resize the masked image to place it where we want it on the background
Finally we can add some fade cues to make the circular image recede and fade. We use separate fades for the opacity and scale so they can use different curves
The sequence then looks like this:
You can download this workspace here
Before we combine all the techniques the final method to explore is blend modes.
In this example we will composite some ripped graph paper over a background using an overlay blend
Here are the 2 images
Surface and cameras settings:
All the surfaces have the same dimensions.
There is a conventional surface called Projector with the projector screen assigned
There are 2 Syphon surfaces:
Main Image which is the surface the image is composited on.
overLay Image which is a syphon surface which a syphon client within the Quartz Composer patch which does the blend will use
And here are the cues:
Background Sent to Main Image Syphon Surface.
(It also has color controls selected in the Video Effects tab for fine tuning)
Graph Paper sent to overLay Image Syphon surface.
(Again, color controls in Video Effects )
Camera Cue with Main Syphon surface fed to Projector with add mode Quartz composer custom composition in video effect tab.
The Quartz composer patch is fairly straightforward and has no controls:
The Syphon Client in the top left is set up to receive the image from the overLay Image Syphon surface. i.e the graph paper
The image port below it receives the video from the camera cue the video effect is in.
The Overlay Blend Mode node uses the syphon client as the image source and the image from the cue as the background and after blending sends it to the output image node.
The node either multiplies or screens the source image samples with the background image samples, depending on the background color. The result is to overlay the existing image samples while preserving the highlights and shadows of the background. The background color mixes with the source image to reflect the lightness or darkness of the background.
Here is the Projector output:
You can download the workspace here
A Complex Composition
We now have all the methods we need to create a composition involving a background, a masked camera cue, and a foreground overlay with the final result given a soft edged border which will look like this:
Here’s the video settings for the workspace:
We have a main video surface which has a mask applied on the surface, and 4 syphon surfaces (the same pixel dimensions as the main surface) which we will use to route video through to achieve the composition.
Our source camera is in the camera patch at position 1. The 4syphon surfaces are in positions 2 -5.
Here are the cues
The background is output to the Main Image Syphon surface
It has color controls applied in the video effects tab for fine tuning.
The camera feed has the camera you want to use selected and is sent to the color correct syphon surface. It also has color controls assigned in the video effects tab
The Mask Camera cue uses the color correct syphon surface as its camera (This has the color corrected camera video on it) and outputs to the Main Image syphon surface. The custom geometry in the tab place the masked video scaled and positioned on the output screen
It has the circle mask custom composition applied in the video tab, with the controls to select the area of interest within the camera feed and to remove the unwanted part of the picture using a circular mask.
Here’s a reminder of the Quartz Composer patch for the mask
The graph paper image is in a video cue and is sent to the overlayImage Syphon surface. It also has color controls applied for fine tuning
The Blending Camera Cue takes the Main Image Syphon surface as it’s input and outputs to the Final Image Syphon surface.
It has the add mode custom composition applied in the video effects tab
This composition picks up the overlayImage Syphon surface (which has the graph paper image on it from the Image to Blend over cue.) and overlays it on the camera feed for this cue (which currently has the Background and main camera images pre composited on it)
Finally in the Composition output cue the Final Image correction Syphon surface is used as input and is output to the Main surface with final image color correction applied in the video effects tab.
The Main Video Surface has a mask applied to give the whole composition a soft edge (This is a black and white mask).
You can download the Workspace here
Basic Template Cue
Here is a link to a basic Quartz composer patch that is a good starting point for rolling your own effects. It has the required input and output ports and the dimension and crop nodes that will ensure the output is as expected. Download here
The basic patch looks like this.
Here’s an example of how you might edit it to provide a simple filter
In Quartz Composer click the Patch Creator button
In the Patch creator window that opens your main area of interest will be patches of the type Image/Filter. In this example we will use Edge Work. Drag the edge work patch to the editor window and connect it between the image input port and the crop node like this
Edge work has one control called radius. We will want to include this as a slider in the QLab video effect tab when the effect is used as a custom composition.
To do this right click on the Edge Work node and publish inputs/radius. right click again and insert input splitter/radius. This will create a node called radius.
Open the patch inspector in the editor window, click on the radius node and in the inspector, select settings on the pop up menu.
Input the range you want for the slider. For this filter a min of 1 and a max of 100 will work well.
Save as Edge Work.qtz. It should now look like this.
You can now use this as a custom composition in the video effects tab of any QLab video or camera cue.
If you edit the patch, e.g to change the max value to 200, save the patch after making the change. To force QLab into using the new version go to the Displays and Geometry tab in the cue and click on the video surface selector (without changing its value). The effect should update.
A Finished QC Set
As the grand finale of this chapter, we’ll now distill all the techniques above into a workspace that uses only 2 custom Quartz Compositions.
The first allows 1 of 6 mask shapes to be selected, including a mask that can receive input from any video (or camera cue) in the workspace.
The second allows 1 of 21 different blend modes to be selected.
Mask Selector QC
The Mask selector QC is an improved version of the circle mask we used earlier.
To make it more useful we are going to add separate controls for x scale and y scale of the mask and add a master scale, so that the overall scale can be adjusted while retaining the x/y scale ratio.
There are 2 maths expression nodes which are set to a*b. This allows the value that the x and y input splitters receive from QLab to be multiplied by the Master Scale value.
In the circle mask QC we only had 1 mask image set to have an embedded grayscale circular mask.
We’ll expand this to 5 different shapes and provide a menu control in the video effects tab of QLab to select which shape to use.
When given blur and rotation controls, these provide a good selection of mask shapes
In addition there is a syphon input that will use anything on a QLab surface that is named syphonMaskFeed as a mask. This can be a grayscale still or even a moving image, so animated masks can be achieved.
In the Quartz Composition we provide the means of selecting which mask to use, using a multiplexer node.
Each of out mask image nodes is connected to a multiplexer port and the source index parameter is published as ‘shape’ and it’s input split to make it available to QLab in the form of a menu.
We set the names we are going to use in this menu in the node settings of the shape input splitter.
This appears with all the other input splitter node names in the controls of the video effects tab in QLab for any cue that has this QC custom composition applied.
I have numbered each splitter node in Quartz Composer as part of their names to force the controls to display in a sensible order instead of alphabetically.
Blend Mode Selector QC
A similar method is used with the blend mode selector which is based on the add mode.qtz composition we used earlier. We use a couple of multiplexers with 2 inputs to allow the foreground and background images in the blends to be swapped and connect the outputs of these to every blend mode available in the standard patch set.
We then use a multiplexer with 21 sources and connect the output of each of the blend mode nodes to 1 of the source inputs.
The source index is published and it input split and all the indices named as before in the settings . In Qlab the blend controls in the video effects tab of any cue that has this composition applied looks like this
In the example QLab workspace provided below the following surfaces and camera patches are used.
Here’s the cue list
The stone image is sent to the composition syphon surface as the background
A video cue of driving past flats at night is sent to the overlayImage Syphon surface which feeds the syphon node in the Blend selector Quartz composer patch which is used in the camera cue below..
A Camera Cue has the composition surface assigned as its camera (which currently has the stone image on it) and has the blend selector QC applied as a custom composition in its Video effects Tab. This cue outputs to the surface named finalColorCorrect
A video cue with a space image still is sent to the same surface but is masked using another instance of the mask selector composition applied in its video effects.
Finally a camera cue uses the finalColorCorrect surface as its input, has some color controls applied in its video Effects tab and outputs to the Projector Output surface.
This looks something like this:
Here are some more compositions using camera and video cues which again have been achieved by using just the 2 Quartz composer patches in the above example.
You can download a file which contains the workspace, the 2 Quartz Composer patches, and the footage used in the example here.