For the last year I’ve been exploring visual-programming for real practical applications.
My initial idea is to give DeepFaceLive (GitHub - iperov/DeepFaceLive: Real-time face swap for PC streaming or video calls) users the ability to modify the logic of program nodes interactively, without resorting to source code.
The conclusions I came to are:
visual programming of simple operations is a failure. 2-3 lines of text in python, in nodes it will be like an entire screen of spaghetti nodes. It’s pretty, but not practical.
visual-programming limits the speed of writing programs. By reading text, we load a larger volume of algorithms into our imagination. When reading nodes, we first take a long time to mouse and scroll, memorize the diagram, then translate it into our internal representation of the algorithm in our imagination. As a result, we produce a lot of redundant actions in the brain compared to reading text code.
Nodes should contain large programs. The user only guides the logic of the data.
Therefore, the nodes should be interactively programmable in a lightweight language like python.
In this case the system can provide 3 types of programming:
No-Code. The user downloads ready-made apps and uses them.
Low-Code. The user makes programs using ready-made nodes, or modifies downloaded apps, adding functionality to suit his needs, such as outputting an image to a stream or saving a sequence of images in a folder.
Full-Code. Same goals as above, except the user programs nodes interactively using hot-reloaded text inside the same studio.
The more abstract the node system is, the smaller the range of tasks it can accomplish. The greater the range of tasks a node system can accomplish, the more it is like directly writing lines of code, but with Basic(lang) flowcharts, which again are difficult to read compared to text code.
A vertical node system can accommodate more nodes and requires mouseover on each node to read the I/O.
Horizontal is easier to read but requires frequent horizontal scrolling.
The advantage of node system over text code is that we can see the result of any function execution interactively. It is like Debug mode of code execution, only the result of any line of code is available at any moment by simply pointing the mouse.
A node system that has a fixed pipeline and has fixed views for users, such as buttons, text and numeric fields, is not difficult to implement.
The pain begins when the code produces dynamic interactive views waiting for input from the user.
For example, selecting the type of Neural Network model in a ComboBox, and depending on that, different settings appear below. Or some settings that spawn other settings.
In this case ViewController should be built in a separate window by the program/nodes itself, increasing the size of the program, and mixing Model code with ViewController code.
Is it possible to design such a general-purpose node system that would be both intuitive for the user
(vvvv is not intuitive even for programmer),
able to implement MVC/MVVM code of any complexity, and async/greenlets/multi-thread friendly?
Or is it just fundamentally impossible?