How ISF source code translates to a Vuo node

Abstract

The nitty-gritty of programming Vuo nodes in ISF

To be recognized by Vuo, the ISF source code must have a file with extension .fs containing GLSL code for a fragment shader.

(Optionally, the ISF source code may also have a vertex shader in a .vs file. This is an experimental feature in Vuo and may not work as expected.)

Node metadata

As with subcompositions and other custom nodes, the file name becomes the node class name. For example, an ISF file called me.image.squiggle.fs becomes a node with class name me.image.squiggle.

The keys and values in the JSON-formatted comment at the beginning of the ISF file are translated to the Vuo node as follows.

ISF key Vuo node characteristic Notes
LABEL Title Shown at the top of the node.
DESCRIPTION Description Shown in the Node Library.
CREDIT Appended to description Shown in the Node Library.
VSN Version Shown in the Node Library.
KEYWORDS Keywords Used when searching the Node Library.

Ports

In most cases, the input and output ports on the Vuo node correspond to the items listed under INPUTS and OUTPUTS in the ISF file’s JSON-formatted comment.

ISF key Vuo port characteristic Notes
NAME Internal name Used when saving a composition to file.
LABEL Display name Shown on the node.
TYPE Data type See the next section for details.
DEFAULT Initial/default constant value For input ports only.
MIN, MAX, STEP Suggested minimum, maximum, and step value For input ports only. Used in the input editor.
VALUES, LABELS Menu items For integer input ports with a fixed set of options. Used in the input editor.

If an ISF input has "TYPE"="size", it is turned into two integer input ports on the Vuo node: Width and Height.

If an ISF file provides no way to determine the output image’s size — no input with "TYPE":"image" or "TYPE"="size" — then input ports Width and Height are automatically added to the Vuo node.

If an ISF file lacks an output with "TYPE"="image", an output port called Output Image is added automatically to the Vuo node.

One Vuo input port is unusual in that it’s not determined by the INPUTS and OUTPUTS (or lack thereof) in the JSON-formatted comment, but rather by the content of the GLSL code. That is the Time port. In any ISF shader, a uniform called TIME of type float is automatically declared. If you use the TIME uniform anywhere in your GLSL code, an input port called Time is added to your Vuo node automatically.

Data types

Vuo supports most ISF data types plus some additional data types specific to Vuo.

ISF data type Vuo data type Vuo-specific?
event Boolean no
bool Boolean no
long Integer no
float Real no
color Color no
image Image no
point2d 2D Point no
point3d 3D Point yes
point4d 4D Point yes
colorDepth Image Color Depth yes
size Converted to two Integer ports yes
bool[] List of Boolean yes
long[] List of Integer yes
float[] List of Real yes
point2d[] List of 2D Point yes
point3d[] List of 3D Point yes
point4d[] List of 4D Point yes
color[] List of Color yes

Output image size and color depth

If the Vuo node created from an ISF shader has input ports Width and Height, the output image’s size is set by these ports. Otherwise, the output image’s size is the same as the image in the first populated image port — in other words, the top-most image port whose popover shows a value other than (no image).

If the Vuo node has an input port of type Image Color Depth, the output image’s color depth is set by that port. Otherwise, the output image’s size matches the image in the first populated image port.

Coordinates

Although not part of the ISF 2.0 specification, to be consistent with many official and unofficial examples of ISFs, Vuo treats inputs of type 2D point specially. If an input has type 2D point and does not have MIN and MAX specified, then the input port value is scaled from normalized coordinates to pixel coordinates when used as a uniform in the GLSL code. For example, if an input port has value (1.0, 0.5) and the output image is to be 1000 x 800 pixels, then the uniform has value (1000, 400).

3D and 4D points are not scaled.

Examples

The examples below focus on how ISF source code translates to Vuo node characteristics, with minimal GLSL code. (For examples with more interesting GLSL code, see the ISF website.) After each ISF source listing is the Vuo node that it creates.

      /*{
          "LABEL":"Make Red Image"
      }*/

      void main()
      {
         gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
      }
      

      /*{
          "ISFVSN":"2.0",
          "TYPE":"IMAGE",
          "LABEL":"Make Opaque Color Image",
         "INPUTS":[
            {
               "NAME":"fill",
               "LABEL":"Fill Color",
               "TYPE":"color",
               "DEFAULT":
               {
                  "r":0.0,
                  "g":0.0,
                  "b":1.0,
                  "a":1.0
               }
            },
            {
               "TYPE":"size"
            }
         ],
         "OUTPUTS":[
            {
               "NAME":"colorImage",
               "TYPE":"image"
            }
         ]
      }*/

      void main()
      {
         gl_FragColor = vec4(fill.rgb, 1.0);
      }
      

      /*{
          "LABEL":"Replace Red Channel",
         "INPUTS":[
            {
               "NAME":"inputImage",
               "TYPE":"image"
            },
            {
               "NAME":"red",
               "TYPE":"float",
               "MIN":0.1,
               "MAX":0.9,
               "DEFAULT":0.5
            }
         ]
      }*/

      void main()
      {
         gl_FragColor = vec4(red, IMG_THIS_NORM_PIXEL(inputImage).gba);
      }
      

      /*{
          "LABEL":"Blend Image Components",
         "INPUTS":[
            {
               "NAME":"image1",
               "TYPE":"image"
            },
            {
               "NAME":"image2",
               "TYPE":"image"
            },
            {
               "NAME":"blendType",
               "TYPE":"long",
               "VALUES":[0, 1],
               "LABELS":["Darker Component", "Lighter Component"]
            }
         ]
      }*/

      void main()
      {
         vec4 color1 = IMG_THIS_NORM_PIXEL(image1);
         vec4 color2 = IMG_THIS_NORM_PIXEL(image2);
         gl_FragColor = (1-blendType) * min(color1, color2) + blendType * max(color1, color2);
      }