Event.VIDEO_FRAME and Camera.copyToByteArray()

Today I got the amazing news that flash player 11.4 beta has been released. And when I read Thibault Imberts blog I was very glad to see that Camera.copyToByteArray() and Camera.copyToVector() are finally out. And not only that but Event.VIDEO_FRAME is back!

Finally! No need for BitmapData.draw() on a Video and fitting the stage.frameRate to your camera fps anymore!

All that’s necessary is the following:

camera.addEventListener(Event.VIDEO_FRAME, videoFrameHandler)
...
protected function videoFrameHandler():void {

    camera.drawToBitmapData(bitmapData);
    texture.uploadFromBitmapData(bitmapData, 0);

    //or
    byteArray.position = 0; // don't forget that!
    camera.copyToByteArray(inputRect, byteArray);
    texture.uploadFromByteArray(byteArray, 0, 0);

    ... 
}


As you can see I’m using a BitmapData to upload the texture. That might be not the best way. You can use texture.uploadFromByteArray() wich seems the much more elegant solution. The reason I didn’t do that is just because I wanted to keep my 640×480; the camera image neither quadratic nor cropped. That’s all. But if you want to use texture.uploadFromByteArray() don’t forget that AGAL textures are written in BGRA not ARGB! You will have to set the endian of the used ByteArray properly like this:

byteArray.endian = Endian.LITTLE_ENDIAN;


Live Demo

Event.VIDEO_FRAME and Camera.copyToByteArray()

You need the latest flash player beta (11.4) to see the live demo

Get Adobe Flash player

If you wonder about those rectangles in my demo ... well it's the same mc but with one difference. The red one is just playing with the regular frameRate of the stage (1 fps - you can change the frameRate by clicking on the upper/lower half of Mr. Doob's Stats ... but you know that, right?). The blue one is rotating with the frameRate of the webcam. I just used gotoAndStop() in the videoFrame event handler. You will see that as soon as you activate the camera.

You can even use Camera.copyToByteArray() and use a regular Bitmap instead of Video.attachCamera() (even if that might be no good idea since we have now the possibility attaching the Camera to StageVideo! ;)). Anything get's updated right then - despite of the value set to stage.frameRate. Wonderful!

If the same would be possible for any Video ... that would be awesome! But never the less this is great news for anyone interested in augmented reality, webcam toys and real-time post-processing.

I haven't tested it on mobile yet, but I'm confident. And if it won't work, just remember Eugene Zatepyakin, whom I'd like to thank right now for pushing this topic.

9 Gedanken zu „Event.VIDEO_FRAME and Camera.copyToByteArray()

  1. Thibault Imbert

    Very nice! Actually I forgot to mention in my blog post and release notes. You can actually even remove the copyToPixels call :) You can use also the brand new Camera.drawToBitmapData :)

    Let me know!

    Thibault

    1. Michael Artikelautor

      Great!

      I just ran a little (and unscientific) performance test. Using a BitmapData is twice as fast as the ByteArray variation? That’s very surprising. It improved a lot in the last months. I did something wrong, didn’t I? :D

  2. Pingback: concurrency: Worker-klasse im FP 11.4 - Flashforum

  3. Eric

    Hello,

    I’m trying to use the same approach on an AIR3.4 Desktop Application.
    Using a Camera instance only, Event.VIDEO_FRAME does not fire. Does it still have to be attached to a Video instance ?

    1. Michael Artikelautor

      Hi guys!

      Yes that’s right. Otherwise flash will not (show the security panel and) connect to the camera.
      [edit]Event.ACTIVATE is fired but nothing else happens. Camera LED does not turn on.[/edit]

      Just for curiosity I tried to use a NetStream instead. In this case the light at my camera indicates that it is active and also drawToBitmapData() is working (eg. in an enterFrameHandler) but the VIDEO_FRAME event doesn’t fire.

      Sorry, I feel I’ve been misleading you. Even if you do not show the video instance in the displayList it seems that you will need the instance to use this event and get the most out of drawToBitmapData().

      Kind regards

      Michael

  4. theguaz

    hey, i’m developing an AR ipad app and today i tried to use that in a camera for a starling class, and i have a very dumb question. DO i have to attach the camera to a video objecto in order to get the Event.VIDEO_FRAME and use the Camera.drawToBitmapData() ?

    regards!

Kommentare sind geschlossen.