Starling WebcamVideo extension

Based on my research I’ve written this Starling extension some weeks ago. I haven’t written on my blog about it, yet, since I had an extension for the Video class in the making as well. I wanted to kill two birds with one stone and “unfortunately” lost track of time writing filters, when I created a real life demo. Thanks for your patience. ;)

As we’ve seen, pushing Camera images up to the GPU can be tricky, but under rare conditions necessary. At least two come to my mind:

  • Augmented Reality. (There is no transparency for Stage3D possible which would allow to use a StageVideo instance behind it.)
  • Real Time Image Manipulation. (Shaders Rock!)

The goal I wanted to achieve with this class is to provide Starling the fastest way possible in pure AS3, for uploading camera images. Usage is pretty simple but if you want to know more about the details, I recommend you to read my previous articles. It’s not a magic bullet, so use with care! If your project has nothing to do with the two points above, I’m pretty sure you’d be better off without this extension.

Starling WebcamVideo live demo

My beard needs some trimming, so no preview image for now. ;)

start demo

webcam chat demo (more on that and the filters soon)

Starling WebcamVideo Example Code

Basically all you have to do is passing the Camera to the WebcamVideo and add it to the stage. For example your Starling class could look like this:

package
{
	import de.flintfabrik.starling.display.WebcamVideo;
	import flash.events.TimerEvent;
	import flash.media.Camera;
	import flash.utils.Timer;
	import starling.display.Sprite;
	import starling.events.Event;
	import starling.events.ResizeEvent;
	import starling.text.TextField;

	public class StarlingWebcamVideoExample extends Sprite
	{
		private var webcamVideo:WebcamVideo;
		private var statsTextField:TextField;
		private var statsTimer:Timer = new Timer(1000, 0);
		private var camera:Camera;

		public function StarlingWebcamVideoExample()
		{
			addEventListener(Event.ADDED_TO_STAGE, addedToStageHandler);
		}

		private function addedToStageHandler(e:Event):void
		{
			removeEventListener(Event.ADDED_TO_STAGE, addedToStageHandler);
			camera = Camera.getCamera();
			camera.setLoopback(false);
			camera.setMode(12800, 7200, 15);
			webcamVideo = new WebcamVideo(camera);
			addChild(webcamVideo);
			statsTextField = new TextField(200, 100, "CAMERA DEBUG", "Arial", 10, 0xFF00FF, true);
			statsTextField.hAlign = "left";
			statsTextField.vAlign = "top";
			statsTextField.y = 26;
			addChild(statsTextField);
			statsTimer.addEventListener(TimerEvent.TIMER, statsTimer_timerHandler);
			statsTimer.start();
			stage.addEventListener(ResizeEvent.RESIZE, resizeHandler);
			resizeHandler();
		}

		private function resizeHandler(e:ResizeEvent = null):void
		{
			if (e)
			{
				webcamVideo.height = e.height;
				webcamVideo.scaleX = webcamVideo.scaleY;
				webcamVideo.x = (e.width - webcamVideo.width) * .5;
				webcamVideo.y = (e.height - webcamVideo.height) * .5;
			}
			else if (stage)
			{
				webcamVideo.height = stage.stageHeight;
				webcamVideo.scaleX = webcamVideo.scaleY;
				webcamVideo.x = (stage.stageWidth - webcamVideo.width) * .5;
				webcamVideo.y = (stage.stageHeight - webcamVideo.height) * .5;
			}
		}

		private function statsTimer_timerHandler(e:TimerEvent):void
		{
			statsTextField.text = "FPS:\t" + camera.currentFPS.toFixed(1) + "/" + camera.fps.toFixed(1) + "\ncam:\t" + camera.width + "x" + camera.height + "\ntextureClass: " + webcamVideo.texture.root.base + "\ntexture:\t" + webcamVideo.texture.root.nativeWidth + "x" + webcamVideo.texture.root.nativeHeight + "\ndraw:\t" + webcamVideo.drawTime.toFixed(2) + " ms" + "\nupload:\t" + webcamVideo.uploadTime.toFixed(2) + " ms" + "\ncomplete:\t" + (webcamVideo.drawTime + webcamVideo.uploadTime).toFixed(2) + " ms";
		}
	}
}


 

Of course you should check beforehand whether a Camera is available etc. … but look who’s talking. I didn’t do that either in my demo, so I’ll leave that up to you. ;)

Recording Area

If you’re using Flash 11.7 / AIR 3.7 or lower, NPOT-textures/RectangleTextures are not supported. In that case it’s best to crop the camera image, as explained in the foregoing articles. To do that you can simply pass a Rectangle. This will “zoom” the image, but can increase performance tremendously.

webcamVideo = new WebcamVideo(camera, new Rectangle(32,36,256,128) );

Note: The width and height of this rectangle are specifying the Texture dimensions, so POT values are expected.

 

Strategy

The default draw/upload-strategy is 0 which means DRAW_BITMAPDATA|OPAQUE|UPLOAD_FROM_BITMAPDATA or to put it into other words, the bitmapData has no alpha channel; it will get populated with bitmapData.draw(video); and the texture will get uploaded directly from the BitmapData, not a ByteArray. According to my tests, this should be fine for most projects.

Anyway, you might want to choose another strategy, for example if you have to process the data in from of a ByteArray. If so, you can always change the strategy by setting the fourth constructor argument like this:

webcamVideo = new WebcamVideo(camera, null, true, WebcamVideo.DRAW_CAMERA|WebcamVideo.UPLOAD_FROM_BYTEARRAY);

 

which means camera.copyToByteArray() will be used and the ByteArray will get uploaded. (Since no BitmapData is used, you won’t have to specify neither WebcamVideo.ALPHA nor WebcamVideo.OPAQUE) However, since a ByteArray is used to upload, you now can access it via webcamVideo.byteArray which otherwise returns null.

Video Events

If you want to access the BitmapData or ByteArray, I recommend not to do this on every frame. Listen for the VideoEvent.UPLOAD_COMPLETE or VideoEvent.DRAW_COMPLETE instead, which will be dispatched by the WebcamVideo instance, to reduce redundant processing. Despite whatever you have in mind to do with the data, this is also a great way to update a cached filter.

webcamVideo.addEventListener(VideoEvent.UPLOAD_COMPLETE, uploadCompleteHandler);
...
function uploadCompleteHandler(e:VideoEvent):void{
    var wv:WebcamVideo = WebcamVideo(e.currentTarget);
    if( wv.filter && wv.filter.isCached) wv.cache();
}

 

By the way: You could even go further and manually pause() the webcamVideo’s recording and listen for the VideoEvent.VIDEO_FRAME to control it via draw() and upload() yourself. But that shouldn’t be necessary.

Scout Preview

webcamVideoScout

Just in case you want to take a look at it, I enabled the advanced telemetry in the demo. Have fun.

Download

You can download the sources from GitHub: Starling-WebcamVideo

4 Gedanken zu „Starling WebcamVideo extension

    1. Michael Artikelautor

      Hi Jeff!

      Sorry, I guess that wasn’t clear. If you’re using a flashplayer version below 11.8 (so without RectangleTexture support), you most likely have to use the rect property of the contructor, like you did. What it does is specifying the size of the texture, the image will get drawn to. Therefore the width and height of this Rectangle have to be the power of two.

      [edit]I changed that, so it will automatically take the next power of two. That way the result may not be as expected, but no error should be thrown.[/edit]

  1. Pingback: Away3D + Starling + Gyroscope on iPhone 4S using AIR 3.9 | everblind.me

  2. Pingback: Starling extensions for video and webcam-video | Flash Talk – All Things Flash

Kommentare sind geschlossen.