Now that the new HTML Texture is out let’s talk about further development to make it industry proof:
Previously on the new HTMLTexture announcement (the one outputing IImage)
Current bugs:
-
Colors messed upFixed in latest alpha -
Context menu appears on right click. This should be an easy fix as CEF exposes context menu event handlers and you can modify or totally clear the context menu.Fixed in latest alpha
In ChromiumFX it’s done this way:
C# code:
var client = new CfxClient();
...
var ContextMenuHandler = new CfxContextMenuHandler();
ContextMenuHandler.OnBeforeContextMenu += (sender, args) => args.Model.Clear();
...
client.GetContextMenuHandler += (sender, args) => args.SetReturnValue(ContextMenuHandler);
I’ll be using ChromiumFX onwards because at first impressions I’ve found it easier to use than CefGlue. And it has a “build with your own CEF build guide”.
Features and node structure
For a project we currently have at MESO we needed 2 features where you need to recompile CEF at the time of writing: h.264 codec support and real multitouch support on any third-party html document.
At first I was trying to work with the vanilla HTMLView from addonpack, but I found that for our purposes it would be cleaner to work with a concise plugin project. So I moved on to modify the HTMLTexture contribution by Gumilastik and Tonfilm.
My aim there is making it spreadable; supporting multitouch with special CEF build; creating a system which has more nodes, express input operations like object bindings and javascript executions as behavior delegates pattern (similar to layers in DX11), and have other nodes getting information from a per-slice monolithic wrapper object. Also I’m placing that vvvv wrapper object into its own assembly so other plugins can use it for their own purposes.
I’m writing down here my experiences to provide devvvvs and people here my findings about this topic. My end result won’t be as user friendly as the single monolithic nodes we have right now, but it will allow slightly more professional usage patterns. That’s also a reason why I switched from addonpack.
Multitouch
For multitouch I found this pull request https://bitbucket.org/chromiumembedded/cef/pull-requests/104/touch-processing-for-osr-different-api/diff and in the discussion I found this patch to apply: https://drive.google.com/file/d/1Sh4qrfkjql3OkdKLY08N7BuUcHUyqQIU/view (for branch 3396 (chromium 67)). Now that patch is not correct because of inconsistent folder structure so I needed to fix it (see attached file below) and even when I fixed it I needed to apply one diff myself to make it compile. Compilation of CEF takes roughly 80-90Gb storage space roughly 3-4 hours of git checkout and preparation if you have a gigabit connection and 3-4 hours compile time, while you can’t do much else because it uses all the cpu. So have an old videogame you always wanted to play lying around while you wait and forget iterative development. I won’t write full instructions here because the CEF building guide should be straightforward to follow here: chromiumembedded / cef / wiki / MasterBuildQuickStart — Bitbucket. Now I managed to compile it with the touch features added and initial tests are fine, so I have 70% confidence that it will work with ChromiumFX (which I’ll start re-building today).
I attach my touch-propcodec patch to make it compile so far and some instructions:
touch-propcodec-patch-3396.zip (293.6 KB)
- Make sure you have Python 2.7 in the PATH environment var. Python 3.x won’t work. If you already have Python 3 there, put Python 2 above it, or delete Python 3 folders from PATH
- Follow instructions here until and excluding step 5: chromiumembedded / cef / wiki / MasterBuildQuickStart — Bitbucket
note: it is important to work in C:\code. If you don’t have 100 Gb free space on C you will have hard time fixing all the batch files with hard coded paths. - from the attachment extract the automate folder to C:\code\automate
- from the attachment extract chromium_git\update.bat to C:\code\chromium_git\update.bat
- run that update.bat preferably from a cmd or powershell to see if it run into errors
(this will take 3-4 hours depending on connection, it’ll download a 11 Gb compressed repo and uncompress it + couple of other operations) - Extract everything from chromium_git\chromium to C:\code\chromium_git\chromium recursively and overwrite if asked
- Run C:\code\chromium_git\chromium\src\cef\create.bat
- Follow CEF build instructions from and including step 8
The ChromiumFX repo has a readme section for regenerating function bindings for new CEF releases/shenanigans which seems to be straightforward.
It’s not as straightforward as I anticipated, posted issue: https://bitbucket.org/chromiumfx/chromiumfx/issues/175/building-chromiumfx-with-own-cef-build
well managed to get over the problem.
Other interesting findings
Currently the CEF result image is copied from GPU to memory by CEF and then copied back into GPU from memory by our wrappers. This is extremely inefficient. Thankfully people are working on this issue and we have this pull request https://bitbucket.org/chromiumembedded/cef/pull-requests/158/support-external-textures-in-osr-mode/diff with this demo: https://github.com/daktronics/cef-mixer which can expose a shared texture directly from CEF so it’s not needed to copy anywhere. I’m planning to experiment with this during autumn again for our current project merging my multitouch/proprietary-codec build with this, but if anyone is quicker with this then suit yourselves ;)
Questions:
What is the price of redistributing something with the h.264 playback capability built into it?
wish me good luck and god speed!