Bitmap data type


#1

Is there any consensus on a bitmap data type yet?
there is System.Drawing.Bitmap, but that seems heavier than IntPtr. Something in-between would be ideal for me, e.g. a struct of:

enum PixelFormat {R8G8B8A8=0, R8G8B8X8, R16G16B16A16, R16G16B16A16F....};
IntPtr       Image;
int          Width;
int          Height;
int          Depth;
PixelFormat  Format;
//bool       isReady //or something else i haven't thought of

Ideally dont want to make an actual copy of the data, or to copy it in and out of a Bitmap every time I access it.
Bitmap functions would be sometimes useful (e.g. resize), but for those types of operations, the overhead of filling a bitmap would be anyway worthwhile(?).

I’m quite conscious that a universally accepted bitmap type would be favourable, with all the AsTexture, Resample, blah, blah nodes to match. Then OpenCV, … .

But perhaps (if for some uses), a different bitmap type was required, then making a casting node wouldn’t be too much trouble.

Candidates I guess are:

  • System.Drawing.Bitmap
  • OpenCV’s IplImage
  • Custom struct
  • Implementing the current DirectShow video type

Anybody else have an interest in this?
vux, were you saying IplImage at node?


#2

Also if we use IplImage, would we have to use something like Emgu CV since there isn’t an official .NET version of OpenCV?


#3

i’d vote for iplimage and yes use the one from emgucv. maybe the non generic IImage interface?


#4

ok, with IPL image…

We’d have to create our own definition for the colour order.
the colour channels are anonymous, so we’d probably define RGBA or ARGB (+variants). DX is RGBA right? (i think the actual types are called like ABGR

there’s no 1bit or 4bit image support, or 16F images (8, 8U, 16, 16U, 32U, 32F, 64F)
no assymetrical bit depth support
no assymetrical resolution support (e.g. 4:2:2 YUV)

I can live with all that though. Anybody else?


#5

I would create a small holder for image types personally, iintroduces no latency, and the problem is using iplimage directly you tie your nodes to the mainloop, so need to think about event handling too.

Also a generic Abstract Image holder would provide the option to choose how you store the picture (like IplImage, Bitmap, Your own stuff…)

For events you can either build an event system (so you run in your own thread, push mode), or have a threaded subgraph which evaluates (pull mode). Both have advantages and drawbacks.

I personnaly prefer pull mode as you can have better sync, mainly if you want to have two inputs pins of image type. Slightly harder to implement the subsystem, but not too bad still.

Also need to think how you want to handle pixel format/depth.

For example, cvThreshold doesn’t support single channel 16U (which is the kinect depth format for example). Don’t know why it’s not implemented as it’s fairly trivial but not there. So either need to display an error output or do some auto conversion (i’d go first route, auto conversion is bad imo).