http://asawicki.info/ Graphics programming, game programming, C++, games, Windows, Internet and more...
Entries for tag "video", ordered from most recent. Entry count: 14.
Color Temperature of Your Lighting
In photography, video and all graphics in general there are so much more parameters to consider than just exposure, meaning lighter or darker image. One of them is color temperature, or white balance. It's about what we consider "white" - a frame of reference, especially concerning light source and so all the objects lit by it. It's not real temperature, but we measure it in Kelvins. Paradoxically, lower color temperature values (like 3000K) mean colors that we call "warmer" - more towards yellow, orange or red. Higher temeratures (like 7000K) mean "cooler" colors - more towards blue. Values like 6500K are considered equivalent of a sunlight during the day, while light bulbs usually have around 3000K. Color temperature of your lighting is important when you work with colors on a computer. They recommend to use 6500K light source for that purpose.
I decided to make an experiment. Below you can see photos of part of my room, with a test screen displayed on my monitor, piece of my wall (behind it, supposed to be white) and a piece of furniture (the right part, also should be white). The monitor is LG 23MA73D-PZ, with IPS panel, calibrated to what I believe should be around 6500K (setting Colour Temperature = Warm2).
Left column shows a photo taken in the middle of the night, with lighting by LED lamps having 3000K color temperature. Middle column is the same scene lit by different LED lamps having 6500K. Finally, right column show a photo taken during the day, using only sunlight.
The only remainig variable is white balance of the photo itself. That's why I introduced two rows. Top row show all three photos calibrated to same white balance = 6500K. As you can see, the image on the screen looks pretty much the same on all of them, because monitor emits its own light. But the wall and the furniture, lit by a specific light source, seems orange or reddish on the first photo, while on the other photos it's more or less gray.
Our eyes, as well as cameras can adjust to changing color temperature to compensate for it and make everything looking neutral-white again. So the second row shows same photos calibrated to white balance of the specific light source. Now the wall and the furniture looks neutral gray on all of them, but notice what happened with the image on the screen when light was 3000K - it completely changed colors of the picture, making everything looking blue.
That's why it's so important to consider color temperature of your light sources when working with color correction and grading of photos, videos or some other graphics. Otherwise you can produce an image that looked good at the time of making, but turns out to have some color cast when seen under different lighting conditions. Of course, if you just work with text or code, it doesn't matter that much. It is more important then to just have a pleasant lighting that doesn't cause eye strain, which would probably be something more like the 3000K lamps.
How to quickly convert MKV to MP4 file using VLC?
Do you have a video in MKV file and you can't open it because some program (like Sony Vegas Pro) doesn't support this format? If so, you probably wonder how to convert it into some different format. I just discovered a way to do this.
To understand this method, first you need to know that media file formats are just containers (for example, MKV is Matroska). Each format encapsulates a set of streams, usually one video and one audio stream. Now, each stream is encoded using some specific codec. There may be various codecs used, but for the file I needed to convert, Media Player Classic (my favorite movie player, installed with K-Lite Codec Pack) shows following information after selecting File > Properties:
Video: MPEG4 Video (H264) 720x400 25fps [V: English [eng] (h264 main L4.0, yuv420p, 720x400) [default]]
Audio: AAC 44100Hz stereo [A: aac lc, 44100 Hz, stereo [default]]
MPEG4 Video is the same codec that may be used with MP4 file format! It means we could convert ("repack") the file to just different container format, rewriting streams as-is without actually converting video or audio - which should be very fast (converting a long movie takes only few seconds) and wouldn't cause any quality loss.
To do that, I used VLC media player. This program has its own set of codecs for many video and audio formats, so it doesn't depend on codecs installed in Windows. The player is actually just an overlay on top of a powerful library that can also do different things, like streaming video over network (that's probably where the company name "VideoLAN" comes from) or convert files.
So to convert an MKV file to MP4:
Writing OFX Video Processing Plugin
Computer graphics is a broad topic and GPU-accelerated real-time 3D rendering is just one of its parts. Another one is photo and video processing. Video editing is an art by itself, but from the technical point of view there are various interesting effects that can be applied. While not essentially real-time and often done on CPU, of course it could also be GPU-accelerated and also should be as efficient as possible. There are many video editing programs, from Windows Movie Maker to Adobe Premiere. The one of my choice is Sony Vegas Pro. Such programs come with a bundle of standard effects, but for a graphics programmer what can be more fun than writing own plugin?
Vegas Pro supports plugins in OpenFX (OFX) standard. They are written as native code compiled to DLL library with appropriate interface. I'm learning how to write such plugins. Below is a description of how you can start experimenting with this technology plus my "Hello World" example that performs simple color inversion. Maybe later I will write a tutorial with detailed description of this API. For now, here is C++ source code of my example with Visual Studio 2013 project and compiled binaries (Win32 and x64): OfxTest1.zip (21 KB). Inside Vegas Pro it looks like this:
A brief description of how to start coding such plugins:
c:\Program Files\Common Files\OFX\Plugins\Basic.ofx.bundle\Contents\Win64\Test1.ofx.
Note that examples provided on SourceForge next to includes package are broken. When compiled in Visual Studio, they assert in Debug configuration due to usage of uninitialized variables and stack corruption! Patch for these bugs is submitted for over 2 years (#1 Basic in Examples - uninitialised variables) and the author is not fixing it. My example has all of this fixed.
Rendering Video Special Effects in GLSL
Rendering real-time, hardware accelerated 3D graphics is one aspect of computer graphics, but there are others too. Recently I became interested in video editing. I wanted to add some special effects to a video and was looking for a technology to do that. Of course video editing software usually has some effects built-in, like different filters or transition effects, some borders or gradients. But I wanted something different. If I had and I knew how to use software like Adobe After Effects, I'm sure that would be the best and easiest way to make any effect imaginable. But as I don't, I decided to use what I already know - to write a shader :)
1. To run a shader, some hosting app is needed. Of course I could write one in C++, but for the purpose of this work it was enough to use Live Coding Compo Framework (a demoscene tool created by bonzaj, which was used during last year's WeCan demoparty). This simple and free package contains rendering application and preconfigured Visual Studio solution. Having VS installed (it works with Express version as well), all I needed to do was to edit "Run.bat" file to point to directory with VS installation in my system. Next, I just executed "Run.bat", and two programs were launched. On the left monitor I had fullscreen "Live Coding Preview", on the right: Visual Studio with special solution opened. I could then edit any of the GLSL fragment shaders contained in the solution. Every time I hit Compile (Ctrl+F7), the shader was compiled and displayed in the preview.
2. Being able to render my effect in real-time, next I needed to capture it to a video. Probably the most popular app for this is FRAPS. I ran it, set Video Capture Settings to frame rate that I was going to use in my final video (which was 29.97 fps) and then captured appropriate period of time of rendering my effect, starting and stopping recording with F9 hotkey.
3. Video captured by FRAPS is in full, original resolution and encoded with some strange codec, so next I needed to convert it to desired format. To do this, I used VLC media player. Some may think that it's just a video player, but in fact it's incredibly powerful and flexible video transmitting and processing software. (I once had an opportunity to work with libVLC - its features exposed as C library.) Its greatest advantage is that it has its own collection of codecs, so it doesn't care whether you have appropriate codecs installed in your system. To convert a video file, I selected: Media > Convert / Save..., selected my AVI file captured by FRAPS, pressed "Convert / Save" button, selected Profile: "Video - H.264 + MP3 (MP4)", customized it using "Edit selected profile" image button, selecting: Encapsulation = MP4/MOV, Video codec = MPEG-4 (on Resolution tab, I could also set new resolution to scale the content, my choice was 1280px x 720px), Audio disabled, Subtitles disabled. Then after pressing "Save", selecting path to destination file, pressing "Start" and waiting some time, I had my video converted to more standard MPEG-4 format (and more than 5 times smaller than the original one recorded by FRAPS).
4. Finally I could insert this video onto a new track in my video editing software and enable blending with underlying layer to achieve desired effect (I used "Overlay" blending mode and 50% opacity).
There are some details that I intentionally skipped here (like video bitrate) not to make this post even longer, but I hope you learned something new from it. My effect looked like this, and here is the source code: Low freq fx.glsl
By the way, here is another tutorial about how to make GIF like this from a video (using only free tools this time):
1. To capture video frames as images, use VLC media player:
2. To merge images into animated GIF, use GIMP:
Sunday in Warsaw - a Time-Lapse Video
I've made my first time-lapse video today. This is a technique in which you capture an image every several seconds, merge these images into a video and the result looks like the time is moving very fast. Here it is:
To make it, I used my webcam connected to my laptop (because USB cable was to short to connect it to my desktop PC :) Of course a special program had to be used to utilize webcam this way, taking snapshot to a JPEG file every several seconds. The program I used is booru WebCam 2.0. It does a good job and it's free, opposite to many others tools of its kind. The result of this program working for several hours was 805 MB of files named like "image-4887.jpg". I had to use another program to convert them to a video.
This can be done with powerful and free VirtualDub. It's not so obvious how to do it though, because there is no "Import images" command in the menu. Instead, one have to know that it's enough to issue a standard File / Open video file command and open first image with it. VirtualDub automatically recognizes subsequent images as video frames. Now the only thing that remained was to set Video / Frame Rate, Audio / Audio from other file, setup filters (resize in my case), video codec and encode final video with File / Export as AVI.
Generating and Compressing AVI Video Files
Yesterday I've decided to share all my home code using Project Hosting on Google Code, so now everyone can browse my code online by entering project called blueway. I'll be very glad to hear your opinions and suggestions about it.
Today I've researched subject of generating video data as AVI file, compressed on the fly with codecs installed in Windows. After succeeding with that I've created simple class to support this task, called VideoCompressor. You can find it in Video.hpp and Video.cpp. Test code is in ProgramMain.cpp, line 2318. It generates a 5 second video with a white horizonal line moving from bottom to top.
Here are some technical details. Functionality needed to handle video files is already inside Windows API, described in MSDN Library. The AVIFile library contains functions like AVIFileOpen, AVIFileCreateStream, AVIStreamWrite. It supports reading and writing AVI files. You need to include <Vfw.h> header and link with Vfw32.lib file. AVI is actually a container file format made of RIFF data chunks identified by 4-byte FOURCC codes (if you ever loaded 3DS models, you know what I mean). It can contain multiple streams like video and audio, which can be coded in many different formats, compressed and decompressed by codecs.
Compressing and decompressing frames of video with installed codecs can be done with Video Compression Manager library. You can display standard, system dialog window to let the user choose and configure codec with ICCompressorChoose function. Then you can just pass subsequent video frames as uncompressed RGB images to a function like ICSeqCompressFrame and you get a piece of compressed data that you can save to your AVI file.
About Graphics in Modern Games
There are always some "trendy" styles that differentiate professional creations from amateur ones. CollegeHumor made a parody of professional-looking trailer for fictional movie Minesweeper [en] and LIMO cabaret did the same with movie Leszek Balcerowicz - Nieznana historia [pl].
What about games? Today people I'm following on Tweeter posted interesting links about graphics in modern games. First, this comic says much. Dark, brown and gray colors are common in today's games, especially in some genres like action games (opposite to colourful fantasy RPG and arcade games, like Trine). Here is also a funny picture. Second, Kayamon posted on his blog quite serious note about how render target size can be reduced by using only two channels to describe pixel colors. His experiment gave quite good results with screenshots from Gears of War :)
People at forums are usually advised to learn BSP, quadtree or octree as space partitioning data structure, but there are many more interesting structures than these three. This time my choice for home project is KD-tree. It's actually something between BSP and octree. Just like BSP it's a binary tree (each non-leaf node has two child nodes) and optimal splitting plane is estimated each time by special algorithm, but splitting planes are always aligned to one of three main axes and thus each node can be described by an AABB (axis-aligned bounding box), just like in octree.
As my tree is designed to manage objects and not geometry, nothing can be split and some of objects may intersect splitting planes. How to deal with them? I simply assign them to the parent node, so not only leaves are allowed to contain list of objects. To avoid too many small objects intersecting splitting planes to degrade performance by falling into top level nodes (it's called "sticky planes" or something like that :) I adopted "loose octree" idea to my KD-tree. It simply means I extend each node’s bounding box so that each node's children slightly overlap each other and small objects intersecting splitting plane fall into one of the children.
My KD-tree is also dynamic, which means it reorganizes itself as objects get added, removed and moved in the tree. It's actually quite simple. Each time an object is added to a node, that node can be split into two children if it's object number exceeds constant limit. Similarly node can be merged by deleting it's children each time an object is removed from one of its children, if the number of objects in that node and its children drops under constant minimum.
For additional performance, I allocate tree nodes from my own "Free List" memory pool and I keep objects connected to each node as doubly-linked list.
I also came up with an idea how to easily visualize quality of my space partitioning technique. I keep track of the number of tree nodes and objects on each depth level. This way I can tell from these several numbers whether tree is more "tall" or "wide" and whether most of objects stay in leaves instead of some top-level nodes.
Here are some screenshots and a video from my recent code: