Glitch-brush with PixiJS
Hi everyone! I continue to dig into different JS libraries. Today I want to share with you how could you achieve cool looking glitch effects with PixiJS. We will build some kind of glitch-brush— analogue of “Smudge tool” in Photoshop but also with RGB channels splitting effect like in example below.
Task
The idea of this task comes up from my friend AA. I found this idea very interesting. Before I saw examples reproduce in computer and mobile apps but I didn’t see something like this on the web reproduced by JS.
To reproduce this effect I choose <CANVAS>. Basically, according to www.w3schools.com its the HTML element used to draw graphics, on the fly, via JavaScript.
Why <canvas>
One of the major reasons here is speed and efficient graphics drawing. In contrast to any DOM animation types like CSS or SMIL, where we have to translate changes to DOM elements attributes, <canvas> allow us to manage any elements and animations on the fly, simply redrawing everything on canvas every time (depends on fps), that is why element called <canvas>. Also, I find effects, methods and attributes in <canvas> much more solid compared to CSS or SVG because in nowadays which I called “Browsers war” you can’t be sure that your cool animation with backdrop CSS filter will look the same in both major browsers like Safari or Chrome for instance (but they could be easier). With <canvas> we don’t have such problems. And of course <canvas> was created especially to draw graphics.
And now “how”. How could we smudge the picture, what we will need to do that?
The plan
For the start before we will make a glitch effect with RGB channels splitting let’s think how we could reproduce a basic “Smudge tool”.
So, we will need a picture container in <canvas>. Displacement container on the top with displacement filter, which position will be related to mouse position — it will be our displacement brush. And on every mouse movement we will bake whole canvas, capture it — displacement brush and picture beneath into one picture, then after mouse movement, we will again bake whole canvas into one picture and so on and so forth. Also, we will need to create RGB channels splitting effect.
Next big question is — which library will fit for us. Which library can provide for us:
- working with images
- displacement filter
- Colour filters (for RGB channels split)
- Blending filters (for RGB channels split)
And all of this and actually much, much more has… drumroll…
PixiJS
Yep, from all libraries which I googled only PixiJS has all of this and also demos page and online editor which makes much easier to understand how it actually works. Also, my choice was based on popularity, because it means that I will have more examples to study and community for help if I will have troubles to manage something.
The first attempt and fail
My first attempt failed because I didn’t know how textures and redrawing works. This example based on this displacement map demo. Also, I bring wrong methods into code like requestAnimationFrame, PixiJS has tickerfor this and some redundant string, but it is just self-flagellation. Let’s continue. What I tried to do is capture canvas by converting it to a base64 image and place it over an old image which as I understand will be expensive for computer memory and could have a bad influence to treasure user fps. But I didn’t see another way and decided to try. But no luck. I can’t understand how to translate this base64 image back into sprite texture.
After two days of useless struggling I decided to ask community. And then I was lucky because I received answer from a really dedicated and experienced community member ivan.popelyshev who helped me to solve this and further questions.
Here is original topic.
Ivan post another code-snipped based on this demo—Masking + renderTexture. And this is exactly what we need.
What is happening in the code? We could separate the hole code into 3 parts.
Good. But, in this example as you might have noticed we have a problem — our texture runs away from us with every movement of the mouse.
The cause is displacement map or if to be more specific in the wrong background colour of the map and that map don’t have any directions.
What “background colour”, and what “directions”?
Displacement map
I came through many professions including 3d modelling and visualisation. And “displacement map” term sounds very familiar to me. Basically, since first 3d tools appeared we had only two types to displace surface by shader — height map (or bump map) and normal map. What is the difference?
Bump map. Its an image with black and white halftones where black colour means depth and white colour means height. It’s very simple and looks like this in practice
Looks natural? Yes, but no. Because if we will look at the surface from a different angle we could see that our bump it’s only an illusion.
Also, bump map gives you less than a normal map in terms of lightning and bump itself, height.
Normal map. It uses a different way to tell surface about heights, it uses RGB channels as XYZ coordinates, it’s more precise way and tell us about all directions not only about heights, and it’s important to understand because PixiJS using exactly this approach.
These are some good examples of comparing normal and bump maps.
So, we have two options now. Create a height map but with neutral background colour relative to height to avoid XY offset (texture runaway).
Our texture now doesn’t run away from us but, as you can see we have only one direction — diagonal to the right bottom corner.
Not exactly what do we want, because we need to catch mouse direction and turns displacement direction into brush direction. Might be we should try a normal map? To generate normal map we will use this online service
Because with a normal map we have all direction(XYZ), the result is very interesting, but still not what do we want.
It means that we should stick the first version of the displacement map. But how we could change displacement direction?
Detect direction
Thank god in PixiJS it’s really simple, we could change X and Y scale and by that control the direction.
displacementFilter.scale.x = value;
displacementFilter.scale.y = value;To detect direction we will use this code inside onPointerMove() func
Also, let’s use a picture of some cute girl as a texture instead of the grass.
Looks good. We have left only the final part… I feel that already wrote too much, so I have to wrap up.
RGB channels split
First, we need to create three containers for each channel and duplicate all texture inside. Basically, we multiply by three all what we have now.
var rt = [], rts = [], bgs = [], containers = [], channelsContainer = [], displacementFilters = [], brushes = [];
var brush, displacementFilter, bg;As you can see we have a brushes array for the brush sprite, displacementFilters for the displacementFilter etc.
We will make all operation through loops, excluding rare cases. For instance
// CONTAINERS //
var containerRed = new PIXI.Container();
containerRed.position.x = 0;
var containerGreen = new PIXI.Container();
containerGreen.position.x = 0;
var containerBlue = new PIXI.Container();
containerBlue.position.x = 0;
containers.push(containerRed,containerGreen,containerBlue);// LOOP //
for (var i=0, len=containers.length; i<len; i++) {
app.stage.addChild(containers[i]);
brushes.push(new PIXI.Sprite(resources.one.texture));
displacementFilters.push(new PIXI.filters.DisplacementFilter(brushes[i]));
bg = new PIXI.Sprite(rts[0][0]);
bgs.push(bg);
containers[i].filters = [channelsContainer[i],displacementFilters[i]];
containers[i].addChild(bgs[i],brushes[i]);
}
Awesome, “but where our RGB channels?” you could ask. To test them we had to create our future textures for each channel. Now when we have them we finally could include BLEND_MODES to overlap channels with ADD blend-mode and ColorMatrixFilter() to extract channels.
ADD blending mode for each container excluding first one
containers[1].filters[1].blendMode = PIXI.BLEND_MODES.ADD;
containers[2].filters[1].blendMode = PIXI.BLEND_MODES.ADD;By colorMatrixFilter() we will change matrix to create Red, Green and Blue channels.
// CHANNEL FILTERS //
var redChannelFilter = new PIXI.filters.ColorMatrixFilter();
redChannelFilter.matrix = [
1, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 0, 1, 0
];var greenChannelFilter = new PIXI.filters.ColorMatrixFilter();
greenChannelFilter.matrix = [
0, 0, 0, 0, 0,
0, 1, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 0, 1, 0
];var blueChannelFilter = new PIXI.filters.ColorMatrixFilter();
blueChannelFilter.matrix = [
0, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 1, 0, 0,
0, 0, 0, 1, 0
];
To create RGB offset effect while we move the pointer let’s set different anchor values for brushes. There is, of course, several different ways to offset RGB channels but let’s stick with that approach.
brushes[0].anchor.set(0.5);
brushes[1].anchor.set(0.6);
brushes[2].anchor.set(0.4);
That’s all and now changing different parameters and displacement brushes you could achieve really interesting results!
Share, like, repost!
Thank you for reading! Sending all of you positive vibes, kitties! ^_^
