I was wondering what the current status or roadmap is surrounding exporting patches as .dll files and being able to dynamically load them at runtime from another executable that doesn’t need to be recompiled?
In our example we would love to export several patches as .dll “plugins” which all use the same interface. Then we want to dynamically load all the dll files in a folder at runtime.
Is any of this in the works or planned for the near future?
Thanks for an update.
It should already be possible to do this right now. You’ll only have to live with the little „Schönheitsfehler“ that the assembly containing your classes has an .exe file extension instead of .dll - but in its core they only differ that one provides an entry point while the other doesn’t. You should be able to copy that exported exe into your plugins folder, and in your host app you’d load that assembly, enumerate over its types implementing your plugin interface and intantiate them. We should probably setup some example and post it here because for sure there’re some culprits which would be quicker to solve on our end.
Okay, that sounds promising! We don’t care about the file ending, so that would be fine. We just want to be able to bundle different plugins with our main exe without having to export a new exe every time and instead can just add/remove plugins as needed.
An example would be much appreciated.
Thanks for the update!
Here you go. Just realized that since .NET Core an exe always has a corresponding dll file. So we can copy the dll files and can skip the exe files. In any case, we still need a dummy comment in the main patch of each plugin so we’re able to export the patch.
Attached you’ll find a
host.vl - this one would be the main application, loading the plugins from the “plugins” folder. In this example it will show a combo box to select the plugin which should run.interfaces.vl - this is the one file all others reference. It contains the plugin interface (called IPlugin in this example, which has an Update operation returning a Skia layer)pluginA.vl - contains a class PluginA implementing IPlugin by rendering “Hello from A”pluginB.vl - contains a class PluginB implementing IPlugin by rendering “Hello from B”You’ll have to export host.vl, pluginA.vl and pluginB.vl and manually copy the pluginA.dll and pluginB.dll files to a new plugins folder beside the host.exe. I also copied the *.dll.config and *.deps.json files, not sure they’re needed, but won’t hurt.
Tested with vvvv 5.2 and vvvv 6.0 preview (in the preview the string rendering in the dropdown is much nicer).
plugin-system.zip (15.1 KB)
Will this also work for input/output types defined in a Shared.vl document? Will the type names be stable and the host.exe will understand the types defined in a common vl document?
Yes. I called that file interfaces.vl in this example. As long as that “shared” file leads to one dll, the system should be happy, types defined in that file should be usable on pins or wherever you like in the host and the plugin. What would break it if the “shared” files references the main file and thereby creating a cycle forcing the system to emit all files taking part in the cycle as one blob.
I’ve tried to replicate this in our project, but I couldn’t get it to work. Is it possible to use this without exporting the host?
On scanning the DLLs there is an error that is a bit weird because it seems to try to look for a VL document when scanning for types in the DLL:
SpaceMusic.PluginArchitecture.vl would be the interfaces.vl in your example.
I’ve copied these files into the plugins folder next to the main VL document:
Without the SpaceMusic.PluginArchitecture.vl.dll the same error occurs.
The main document references the SpaceMusic.PluginArchitecture.vl, the plugin architecture file also contains the nodes to scan and load plugins, could that be an issue?
EDIT:
just tried this in your example and it seems to have the same problem:
EDIT2:
I’ve tried to load the DLL of the interfaces.vl file instead of referencing the vl document, but loading it creates this error and vvvv doesn’t do anything anymore:
NullReferenceException: "Object reference not set to an instance of an object."
StackTrace:
VL.Lang.Platforms.Roslyn.AssemblySymbolSource+<>c__DisplayClass10_0 { internal VL.Lang.Platforms.Roslyn.CompiledSymbols <.ctor>b__1() { ... } }
System.Lazy`1 { private void ViaFactory(System.Threading.LazyThreadSafetyMode mode) { ... } }
System.Runtime.ExceptionServices.ExceptionDispatchInfo { public void Throw() { ... } }
System.Lazy`1 { private T CreateValue() { ... } }
VL.Lang.Platforms.Roslyn.AssemblySymbolSource { internal VL.Lang.Platforms.Roslyn.CompiledSymbols get_CompiledSymbols() { ... } }
VL.Lang.Platforms.Roslyn.SymbolSourceProvider { public virtual VL.Lang.Symbols.ISymbolSource GetSymbolSource(string location) { ... } }
VL.Lang.Symbols.DocSymbols+DependencySymbol+<GetReferencesCore>d__29 { private virtual bool MoveNext() { ... } }
System.Collections.Generic.LargeArrayBuilder`1 { public void AddRange(System.Collections.Generic.IEnumerable<> items) { ... } }
System.Collections.Generic.EnumerableHelpers { internal static T[] ToArray(System.Collections.Generic.IEnumerable<> source) { ... } }
System.Collections.Immutable.ImmutableArray { public static System.Collections.Immutable.ImmutableArray<> CreateRange(System.Collections.Generic.IEnumerable<> items) { ... } }
VL.Model.InterlockedHelper { public static System.Collections.Immutable.ImmutableArray<> Init(System.Collections.Immutable.ImmutableArray<>& this, System.Collections.Generic.IEnumerable<> values) { ... } }
VL.Lang.Symbols.DocSymbols+DependencySymbol { public virtual System.Collections.Immutable.ImmutableArray<VL.Lang.Symbols.ISymbolSource> get_References() { ... } }
VL.Lang.Symbols.PreCompilation { internal VL.Lang.Helper.Graph<uint, VL.Lang.Symbols.DocSymbols> BuildDocumentGraph(bool skipPreCompiled, bool introduceBackedges) { ... } }
VL.Lang.Symbols.PreCompilation+<>c__DisplayClass19_0 { internal System.Collections.Generic.IEnumerable<VL.Lang.Symbols.IProjectSymbol> <Initialize>g__CreateAdhocProjects|14() { ... } }
VL.Lang.Symbols.PreCompilation { private void Initialize(VL.Lang.Symbols.PreCompilation previousCompilation, System.Threading.CancellationToken token, System.IProgress<VL.Model.LoadMessage> progress) { ... } }
VL.Lang.Symbols.PreCompilation { internal static VL.Lang.Symbols.PreCompilation Create(VL.Lang.Symbols.IPlatform platform, VL.Model.Internal.Solution solution, VL.Lang.Symbols.PreCompilation previous, VL.Model.CompileOptions options, System.Threading.CancellationToken token, System.IProgress<VL.Model.LoadMessage> progress) { ... } }
VL.Lang.Symbols.PreCompilation { internal VL.Lang.Symbols.PreCompilation WithSolution(VL.Model.Internal.Solution value, VL.Model.CompileOptions options, System.Threading.CancellationToken token, System.IProgress<VL.Model.LoadMessage> progress) { ... } }
VL.Lang.Symbols.PreCompilation { public VL.Lang.Symbols.PreCompilation WithSolution(VL.Model.Solution value, VL.Model.CompileOptions options, System.Threading.CancellationToken token, System.IProgress<VL.Model.LoadMessage> progress) { ... } }
VL.Model.VLSession+<>c__DisplayClass194_0 { internal VL.Lang.Symbols.PreCompilation <PrepareSolutionWithFreshSymbolsAsync>b__0() { ... } }
System.Threading.Tasks.Task`1 { internal virtual void InnerInvoke() { ... } }
System.Threading.Tasks.Task+<>c { internal void <.cctor>b__272_0(object obj) { ... } }
System.Threading.ExecutionContext { internal static void RunFromThreadPoolDispatchLoop(System.Threading.Thread threadPoolThread, System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) { ... } }
System.Runtime.ExceptionServices.ExceptionDispatchInfo { public void Throw() { ... } }
System.Threading.ExecutionContext { internal static void RunFromThreadPoolDispatchLoop(System.Threading.Thread threadPoolThread, System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) { ... } }
System.Threading.Tasks.Task { private void ExecuteWithThreadLocal(System.Threading.Tasks.Task& currentTaskSlot, System.Threading.Thread threadPoolThread) { ... } }
System.Runtime.ExceptionServices.ExceptionDispatchInfo { public void Throw() { ... } }
System.Runtime.CompilerServices.TaskAwaiter { private static void ThrowForNonSuccess(System.Threading.Tasks.Task task) { ... } }
System.Runtime.CompilerServices.TaskAwaiter { private static void HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task task) { ... } }
System.Runtime.CompilerServices.TaskAwaiter`1 { public TResult GetResult() { ... } }
VL.Model.VLSession+<PrepareSolutionWithFreshSymbolsAsync>d__194 { private virtual void MoveNext() { ... } }
System.Runtime.ExceptionServices.ExceptionDispatchInfo { public void Throw() { ... } }
System.Runtime.CompilerServices.TaskAwaiter { private static void ThrowForNonSuccess(System.Threading.Tasks.Task task) { ... } }
System.Runtime.CompilerServices.TaskAwaiter { private static void HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task task) { ... } }
System.Runtime.CompilerServices.TaskAwaiter`1 { public TResult GetResult() { ... } }
VL.Model.VLSession+<UpdateCompilationAsync>d__192 { private virtual void MoveNext() { ... } }
The .vl part in the assembly name is ok I think - the file is called Foo.vl.dll I believe. Other than that hard to say what’s going on on your end - the zip file with the instructions I posted works at my end, does it for you?
We figured it out, the referenced DLL wasn’t found because .NET wasn’t looking for that file in that directory. If we copy the common DLL with the interface next to the vvvv.exe it gets found.
So we are one step further, unfortunately, we hit another issue:
This doesn’t work as the vvvv runtime type in the editor (the one we used to annotate the IOBox) seems to be a different type than what the plugins implement and the IsAssignableTo fails… Any idea how we can make sure the system thinks it is the same type?
What do you mean you copy it beside vvvv.exe? I thought this whole endeavor is about loading plugins from an exported app?
Yes correct, but first we need to develop and test. For that, we need a workflow that allows us to load plugins and integrate/test them into the main app. So, we need the plugins to be loaded at vvvv design time.
Later we also need it the other way around, have the exe running and patch plugins…
The plugin DLLs are in the plugin folder. But they can only be loaded if the DLL with the interfaces is also found, see the exception screenshots above. We figured out that if we copy the DLL with the interfaces next to the vvvv.exe we can load the plugin DLLs at design time in vvvv, which is great.
Unfortunately, the system doesn’t recognize the interface from the DLL as the same interface in the VL document. The VL document with the interfaces is referenced in the main app. We annotate the IOBox with that type from the document, but this is then not assignable by the type in the plugin DLL.
So you load the compiled dll from within the editor which has the same “interface code” loaded as a (let’s call it) dynamic dll. For the system these are now two different dlls (the static interfaces dll and the dynamic one), so a type mismatch makes sense.
I think you’d need to kick out the reference to interfaces.vl while you’re editing and replace it with a reference to interfaces.dll - but surely that will not make things any easier.
Maybe think it differently and find a testing workflow where you work with the source code and not a mix of it?
Yes, not super convenient, but a very good idea, it sounds like a feasible way to do it. We export the interfaces and the common types in a small DLL and forward that in the common VL document. Then all cases should use the same types that originate from the same DLL. Assuming that the interface and the common types will rarely change, it is a good trade-off. We’ll try that!
It worked, great stuff:
This looks super useful!
Here is a quick summary:
Working Proof of Concept: We have successfully established a process where interfaces and common types are defined in a class library, compiled into a DLL by vvvv, and then referenced in both the main application and the plugins vl documents. This setup allows us to avoid type mismatches and ensures consistency across the application and its plugins.
Plugin Live Development: To offer a dynamic ‘edit while running’ experience for plugins without the ‘edit-compile-load plugin-run’ cycle, the current proof of concept introduces a challenge. To maintain the live development, the host application (compiled or patched) must be running to allow the development of plugins in real-time within the vvvv editor.
Current Status: Elias helped us to create a project setup that attempted to load the compiled main application in a patch and set a plugin for it at vvvv design time, but we hit a roadblock. While the proof of concept looks very good, we haven’t yet found a seamless solution that supports live development. The goal remains to find a method where plugin developers can use the ‘edit while running’ experience without needing access to the main application’s source code.
