Something like Syphon on Windows (currently called Wyphon)

Hello Benkuper,

glad you are getting closer.

Am I right in assuming that you are developing in C++?

In that case you have defined a callback function that has told you what the handle of the shared texture is, right?

Like so:

void D3DTextureSharingStartedCALLBACK( HANDLE wyphonPartnerHandle, unsigned **int32 sendingPartnerId, HANDLE sharedTextureHandle, unsigned**int32 width, unsigned __int32 height, DWORD format, DWORD usage, LPTSTR description ) { ... }

Now you will need a DirectX device in order to be able to open the shared texture.

Examples for setting up DX11 can be found here:
http://code.msdn.microsoft.com/windowsdesktop/Direct3D-Tutorial-Win32-829979ef

After you have successfully created a device (g_pd3dDevice), you can open the texture using its handle (in DX9ex, you would also need the other parameters, like width, height, format etc.):

ID3D11Texture2D * g_pSharedTexture;

ID3D11Resource * tempResource11;

g_pd3dDevice->OpenSharedResource(sharedTextureHandle, __uuidof(ID3D11Resource), (void**)(&tempResource11));

if (tempResource11 != NULL) {
	tempResource11->QueryInterface(__uuidof(ID3D11Texture2D), (void**)(&g_pSharedTexture));
	tempResource11->Release();
}
//now you can do whatever you want with g_pSharedTexture

Once you have done this, what you do with it is pretty much up to you.

(BTW I should have provided the .h files somewhere, instead having you try to find out what the callbacks should look like by yourself)

.h files for Wyphon and WyphonUtils (1.7 kB)

Hi ft,
thank you for your reply !

Actually, it’s not a pure C++ project because i couldn’t get a sample program to work, as i didn’t (and still don’t) know how to use the callbacks.

So i’ve created a C++ / CLR (.NET) project that calls C# methods. This way, i can get both C++ code and C# code running, and i use the WyphonDotNet lib because this way i understand how to use the callbacks. (sorry if it’s not crystal clear…).

I’ve already looked up in the headers files from the repository, but still couldn’t find a way to create a simple project with simple callbacks… C++ seems so complicated each time you have to do even the tiniest things !

So maybe it would be too much to ask, but it would be really helping if you could post a sample C++ project with all initialisation and callbacks functionning, so i can understand the full workflow.

For the DX11 stuff, i didn’t try anything yet, so i will be quick : Adobe AIR is dealing with OpenGL from what i know, so i won’t be able to access directly the resources it creates. My aim is to be able to share images as bytearrays or bitmapdata between programs so i don’t have to send them through the network.
As you have shown that wyphon is able to share not only textures but also custom datas between programs, i’m trying to find the best way to make it more universal, and be able to easily share bytearrays between programs so it would also be possible for processing, air, unity and other programs to integrate wyphon.

What do you think ?

Thanks again for you work and dedication

@benkuper:

I know I should get some samples online soon. They are on my harddisk, but a bit too messy. I will try to clean them up somewhere in the next few eeks, and get them on github.

It is true that Wyphon is built on something else I wrote, that makes it easy to broadcast messages between applications.
I don’t think that it would be a splendid idea to send whole images though, as 1 message (I am actually using mailslots) would be sent to every ‘partner’ application. That’s cool for small messages, but not for big things like image data.

Another idea could be, that we add some functions to Wyphon, that allow you to easily share information about shared-memory, that contains an image. This way, a ‘sender’ could update that shared memory, and the ‘receivers’ could read that.
But I guess such things exist already when using the right freeframe plugins. Apart from that, performance and frame size will probably always be limited, which is of course why people want to share images on the GPU.

I copied some code as an example of how to setup the callbacks in C++ and D3D11:

//--------------------------------------------------------------------------------------
// Wyphon Callback functions
// 
//--------------------------------------------------------------------------------------

void D3DTextureSharingStartedCALLBACK( HANDLE wyphonPartnerHandle, unsigned **int32 sendingPartnerId, HANDLE sharedTextureHandle, unsigned**int32 width, unsigned __int32 height, DWORD format, DWORD usage, LPTSTR description ) {
	int newIndex = -1;
	for ( int i = 0; i <= g_maxSharedTextureIndex; i++ ) {
		if ( g_pSharedTextureHandles[i](i) == sharedTextureHandle ) {
			//texture already known, do nothing
			return;
		}
		if ( g_pSharedTextureHandles[i](i) == NULL ) {
			newIndex = i;
			break;
		}
	}
	

	ID3D11Resource * tempResource11;

	g_pd3dDevice->OpenSharedResource(sharedTextureHandle, __uuidof(ID3D11Resource), (void**)(&tempResource11));
	if (tempResource11 != NULL) {
		g_pSharedTextures[newIndex](newIndex) = NULL;
		g_pWyphonShaderRVs[newIndex](newIndex) = NULL;

		tempResource11->QueryInterface(__uuidof(ID3D11Texture2D), (void**)(&g_pSharedTextures[newIndex](newIndex)));
		tempResource11->Release();

		// now use g_pSharedTexture with pDevice11 
		if ( g_pSharedTextures[newIndex](newIndex) != NULL ) {
			if ( ! FAILED( g_pd3dDevice->CreateShaderResourceView( g_pSharedTextures[newIndex](newIndex), NULL, &g_pWyphonShaderRVs[newIndex](newIndex) ) ) ) {
				g_pSharedTextureHandles[newIndex](newIndex) = sharedTextureHandle;
			}
			else {
				//cleanup this shared texture?
				g_pSharedTextureHandles[newIndex](newIndex) = NULL;
				g_pSharedTextures[newIndex](newIndex)->Release();
			}
			
		}
	}

}

void D3DTextureSharingStoppedCALLBACK( HANDLE wyphonPartnerHandle, unsigned int sendingPartnerId, HANDLE sharedTextureHandle, unsigned int width, unsigned int height, DWORD format, DWORD usage, LPTSTR description ) {
	//find handle
	int index = -1;

	for ( int i = 0; i <= g_maxSharedTextureIndex; i++ ) {
		if ( g_pSharedTextureHandles[i](i) == sharedTextureHandle ) {
			index = i;
			break;
		}		
	}

	if ( index > -1 ) {
		//if found, empty that spot
		g_pSharedTextureHandles[index](index) = NULL;
		
		g_pSharedTextures[index](index)->Release();
		g_pWyphonShaderRVs[index](index)->Release();
	}	
}

And then somewhere in your initialization method:

///////////////////
	//// SETUP Wyphon
	///////////////////

	for ( int i = 0; i <= g_maxSharedTextureIndex; i++ ) {
		g_pSharedTextureHandles[i](i) = NULL;
	}

	//2 callbacks since we are only interested in textures from other partners, not who those partners are
	g_hWyphonPartner = CreateWyphonPartner( is64bit ? TEXT("Wyphon RECEIVER test 64bit") : TEXT("Wyphon RECEIVER test 32bit")
							, NULL
							, NULL
							, NULL
							, D3DTextureSharingStartedCALLBACK
							, D3DTextureSharingStoppedCALLBACK
	 						);

Hi ft,
thanks a lot for this sample code !
I just came home and got my hands on it again, i understand more how this should be done now.
However, i had some difficulties creating the directX devices and even if it compiles well, i’m not sure i’m doing it right.

When i run the program, it creates the wyphon partner well, and when another program share a texture, it launches the callback, but then i get an “Access Violation Exception at reading 0x00000000” when reaching this line :

g_pd3dDevice->OpenSharedResource(sharedTextureHandle, __uuidof(ID3D11Resource), (void**)(&tempResource11));

From what i read, it could come from a null pointer or uninitialized variable.
I guess it comes from a bad creation of the D3D device…

I joined the C file with all the initialization and handling.

About your ideas for image sharing, i think it’s great, but i’m not sure i understand fully the concept.
My main idea is to have this kind of flow :

1* Program1 in AIR generates BitmapData (ByteArray)
2* Native extension in C (compiled in the Program1) converts the bytearray in D3DTexture and share the HANDLE using Wyphon
3* Program2’s native Extension receives the shared texture info from Wyphon, converts it back to a ByteArray
4* Program2 get the byteArray and generate a bitmap from it

Is that what you think is not a good idea ?

Thank you again for your help, i hope i will be able to have a functional test program soon :)

Wyphon C main file (1.5 kB)

@benkuper:

User WyphonUtils to setup a DX9ex device that can be used to create a texture on first, or find some DX11 examples on the internet to see how to setup the g_pd3dDevice correctly.

About copying the texture back from GPU to memory, that would be a bad idea because it is slow. It would be better if Program2 did everything else it had to do to the image data on the GPU by using shaders.

A c++ project that can serve as an example to get started integrating Wyphon into your own application can be found here:


(also check the wiki for some extra info)

All of the files you should need to add Wyphon support to your own application (including the compiled example programs) are in this github repo:

Some hints on how to use Wyphon in c# can be found here:


and also in the Wyphon nodes for vvvv:

(start with WyphonNode.cs)

Hello guys,
it’s been a while since somebody posted on this thread, but i hope it’s not dead…

I got some time to get a fresh start with the Unity integration of Wyphon, and i got further, and i have good hopes that it will work !

I managed to get the lib working and sharing a texture from Unity (using the C++, not the WyphonDotNet), and the Receiver Test 32bit is reacting to the sharing, as well as the console test app (in the console, the unity wyphon is registered as “sender test 32bit”)

However, i’m only getting a black texture, when i’m trying to share a rainbow-ish one (the plane has the shared texture in the image below).

I have attached a screenshot of the 3 softwares running together, and the .cpp that i use to compile the .dll for Unity.
I took the base project for Unity Native Code Plugins and added the Tutorial2 code. I have deleted a lot of code that is not related to the texture sharing (since Unity handles the rendering). I may have deleted too much…

@ft : I have a question for you ! In the tutorial2, I saw that each frame, you execute the Render function which grab the screen and copy the data in a D3D11Texture2D. Since i already have a D3D11Texture2D from Unity, i’m not sure of the proper actions i should take to get the flow working.
I suspect Unity’s Texture2D may not have the same characteristics than the one necessary for shared textures, but i can’t really go further alone…

Click for original size

Hope you can answer to that,

thanks

Ben

WyphonPlugin.cpp for Unity (3.6 kB)

From the top of my head (it’s been a while), I can tell you that the D3D11 texture in the ‘sender’ demo is created from a D3D9ex texture (you create a shared D3D9ex texture first, and then open the D3D11 texture using the returned handle).

I don’t know if you can just use Unity’s texture, as it has probably not been created as a shared texture, which is why there’s a big chance you should copy the Unity texture to the shared texture that you created yourself (we created WyphonUtils.dll for that, so you don’t have to worry too much about setting up a D3D9ex surface and creating a properly shared texture).

The reason we chose to share D3D9ex textures, is to make Wyphon as ‘open’ as possible (otherwise your partner must be D3D11 too, and now anyone supporting D3D9ex or up can open the shared texture).

Another problem could be that not all of the parameters are right, like width/height/texture format. Some texture formats work well, some seem to give trouble.

Anyway, I hope this information already gives you some pointer to what the cause of the problem might be.

vux eventually mentioned that dx11 texture created with mrt renderer in RGBA16 can be shared with dx9ex, also i don’t know how efficient it is but a libraryA to CUDA/OpenCL to libraryB (so e. g. OpenGL - CUDA - DX11) solutions are floating around on the internet i think both worth to check out

Thank you microdee.

After a talk with frederik and a lot of headache, here’s where i am :

  • Resolume and ft’s Sender Test can send textures to Unity successfully.
  • Unity can send BGRA32 textures to sender test and resolume successfully, using DX9Ex and DX11. The problem is BGRA32 is really annoying to work with in Unity.
  • Unity can send RGBA32/ARGB32 textures to sender test successfully using DX11 only, but the FFGL plugin is handling shared textures with DX9 so if it’s not BGRA32, it doesn’t work.

Right now, i’m trying to recompile FFGL-DX Plugin with full DX11 support.
If someone can help me do that, i think we can pull something pretty fast.

Hi everyone,
just an update : i got Wyphon fully working between Unity and Resolume in both direction !
I have recompiled elio’s FFGL plugin to be DX11 only, so there are less contraints on texture formats.

I will post videos & sources as soon as i cleaned up the project.

Hi everyone,
just an update : i got Wyphon fully working between Unity and Resolume in both direction !
I have recompiled elio’s FFGL plugin to be DX11 only, so there are less contraints on texture formats.

I will post videos & sources as soon as i cleaned up the project.

Hello there !
Just a quick word to say i’ve almost finished the Unity Wyphon Plugin, working both in and out !
A short video that demonstrates the Unity Resolume texture sharing via Wyphon : https://www.youtube.com/watch?v=aFLeTWJK0ho

Source code will come soon.

Many thanks to ft for his work and discussion time, and elio for the FreeFrameGL Wyphon Bridge

Cheers !

@benkuper: wooow!!!

@benkuper

Thanks a lot for your hard work.

Maybe (because I didn’t foresee a mechanism to specify if the shared texture is DX9ex or something else), it would be an idea that you share your DX11 textures with a width, height, format and usage of 0, so that is clear that they will not be readable by any dx9ex application.

I think this is a good compromise, since dx10 and 11 applications only need the handle anyway.
This way we remove the constraint that any application is forced to share DX9ex textures only.
Then it is also allowed to share DX11 textures (which is easier, and less limited), but with the limitation of course that you exclude DX9ex applications from opening these textures.

EDIT: oh and I found this (http://msdn.microsoft.com/en-us/library/windows/desktop/bb172412(v=vs.85).aspx), so I think you should create your textures with the D3D10_RESOURCE_MISC_SHARED_KEYEDMUTEX flag, and lock them whenever you write to them, and whenever you read from them. You would be the first to support this, but it is the proper way in my opinion to guarantee that your texture will not be rendered by the receiver and written to by the sender at the same time (which might cause some flickering from time to time)…

EDIT2: Sorry, I spoke too early, since that would mean anyone should lock the texture before usnig it, and since no-one supports this yet. So maybe the locking support should become something for Wyphon2…