http://asawicki.info/ Programming, graphics, games, media, C++, Windows, Internet and more...
Vulkan 1.0 Released!
Yesterday (2016-02-16) was a big day - Vulkan 1.0 has finally been released. The new 3D graphics and compute API from Khronos Group has a chance to be the solution long awaited in the PC world that will:
Time will tell whether Vulkan becomes popular, common standard. It's not so certain. Microsoft promotes its own Direct3D 12, Apple has its Metal API, NVIDIA develops CUDA, old OpenGL and OpenCL are here to stay. What hardware versions and software platforms will eventually support the new API? What will be the quality and performance of those drivers? Will some good debugging and performance probiling tools become available? Will game developers and game engine developers port their code any time soon? What the reception will be among video/media, CAD/CAM, HPC professionals? I'm very enthusiastic, seeing so many learning materials and code samples available since day one! Just look at #Vulkan and #VulkanAPI hashtags on Twitter.
Some useful links to start with:
How code refactoring can fix stack overflow error?
tl;dr: A very long C++ function with multiple local variables, even if they are not very big and they are placed in separate scopes, can reserve as much as hundreds of kilobytes of stack frame, causing "Stack Overflow" even without bugs like infinite recursion. So you better split your long functions into shorter ones.
Can refactoring (or the lack of thereof) cause application crashes? If we understand refactoring as changes in code layout without changing its logic, we might think that it's just the matter of readability and unreadable code increases chances of introducing bugs. But here is a story in which refactoring actually fixed a bug.
Long time ago in a software project far far away, there was a bug submitted telling that the application crashes with "Stack Overflow" message. It was a Windows app, developed in C++ using Visual Studio. I thought: - I can handle that, it should be easy! Every beginner/intermediate programmer knows about the call stack and surely seen this error at least once when accidentally caused infinite recursion in his code. So my first idea was that infinite recursion happens because of some logical error in the code (that should be easy to fix) or some unfortunate, invalid input data (that should be validated for safety before usage).
As it turned out, this was not the case. After setting up all the test environment and catching the crash in Visual Studio debugger, I looked at Call Stack and noticed that it looks quite normal. Sure the call depth was significant (as for C++, I'm not talking about Java here ;) and there was even some recursion, but 20 or 30 functions is not that much. The stack ended with a call to non-recursive function that seemed correct, so it was not the recursion that caused stack overflow.
My second idea was that some of these functions allocate some big objects (like arrays) by value, as local variables on the stack and this causes the stack to grow too big. I reviewed code of the functions that I found on the stack and used "Immediate Window" panel to quickly check sizeof(xxx) of variables or their types when they used some class, but I didn't find anything particularly big. Local variable sizes varied from few bytes to at most several hundred bytes and I couldn't find any big arrays defined in these functions. I also fetched address of some local variable in a function near the bottom of the stack (which looks like 0x000000000009a370), address of a parameter from the function at the top of the stack and subtracted them to see how big the stack grown over all these calls. The result was around 50 KB - not that much.
My third idea was to check maximum size of the stack. It is 1 MB by default, but it can be changed in Visual Studio project settings, in Linker > System tab, as "Stack Reserve Size" parameter. I check my project and I found this parameter not changed from its default value.
OK, now this became more difficult than I thought. After many debugging sessions, where I looked at various pointers, addresses and numbers trying to spot some memory override, stack corruption, out-of-bounds indexing etc., I finally opened "Disassembly" and "Registers" panels. I'm not a fan of such low level stuff, so it took me some time and few Google queries to understand these RSP, RBP registers and make sense of some x86-64 opcodes. While debugging step-by-step in the assembly, I found something interesting. At the beginning of my function, there was a call to mysterious function
__chkstk and the crash occurred inside it. That was a clue I could use to ask Google what this all means. I found this: Description of the stack checking for Windows NT-based applications and this: What is the purpose of the _chkstk() function? These articles say that as the stack grows, next 4 KB pages are reserved. Each next page is allocated by the system on first "touch". I could actually see in my debugger that functions which need less than 1 page (4096 B = 1000h) have an instruction at the beginning similar to this:
While my debugged function had this instead:
call __chkstk (018104AA00h)
The articles say that when reserving more than one page of stack memory, this function must be called to loop over addresses with 4 KB step and "touch" each page. This is really what it does:
--- f:\dd\vctools\crt\crtw32\startup\amd64\chkstk.asm ---
mov qword ptr [rsp],r10
mov qword ptr [rsp+8],r11
mov r11,qword ptr gs:[10h]
jae cs10+10h (018104AA40h)
mov byte ptr [r11],0
jne cs10 (018104AA30h)
mov r10,qword ptr [rsp]
mov r11,qword ptr [rsp+8]
Key sentence of the second linked article seems to be: "The parameter in rax is size of data you want to add." In my case, eax is set to 26B29h = 158505. Wait, what?! This is more than 150 KB! Is it really how much of the stack the function needs?!
It was finally the right conclusion. The function was more than 3000-lines long, with lots of nested conditions and all kinds of stuff, but mostly an all-encompassing switch with dozens of different cases. I refactored it, extracting code from under each case to a separate function. This fixed the "Stack Overflow" crash.
Apparently if you have a long function and define a lot of local variables, even if they are not particularly big and they are placed inside separate scopes like if-s or switch case-s, the function may need as much as 150 KB of stack frame, at least in Debug configuration. This can cause crash with "Stack Overflow" message even without infinite recursion or bugs like that. So please keep this in mind as additional argument for refactoring your code as soon as you see the need for it.
Global Game Jam 2016 - Postmortem of our project
Last weekend, this year's edition of Global Game Jam took place all around the world. Just like in previous years, I participated in 3City Game Jam - a site in Gdańsk, Poland. It is a big one, with over 150 participants, organized by Playsoft company in their office. Theme this year was "Ritual". Regarding technology, Unity was most popular in our site, with just few games using something else: Unreal Engine, HTML5, GameMaker and C++ with SFML.
We have also used Unity. Our team consisted of 3 programmers. Here you can see our game: Bloody Eclipse, but it is far from being finished or playable. Honestly speaking, in my opinion the project on this jam went exceptionally poor. We didn't even make it to the top 10 best voted games to be presented on a big screen. That's why I'd like to share some conclusions, for you as well as for my future self.
First, it were not environmental issues that caused any problems. We all had our hardware and software set up before the jam, with Unity, Visual Studio, Git client and other tools already in place. Internet worked perfectly with transfer up to 80 Mbps in both directions. Second, it was not a lack of knowledge or skills. Our work in Unity went quite smoothly. We could deal with C#, 3D math and Git pretty well. Third, it was not because of the lack of artists in our team. Sure, graphics is very important for overall experience, but the guys who made The Bad Ritual also didn't have artists in their team and they somehow found a consistent visual style for their game, made it fun and pretty. There are many possibilities to make minimalistic and yet visually pleasant game, just like there are many free assets ready to use in Unity Asset Store.
The biggest thing that was missing in our team was management/leadership. I deliberately don't call it planning or design, because in a hectic environment like a game jam it's not enough to design the game at the beginning and then just execute. Things are changing fast, new ideas come to mind, time is running fast and new obstacles appear (like bugs or difficulties in development), so someone should have an authority to decide what to do next, keep the list of tasks "TODO" and update it constantly with priorities assigned so the most important things are done first. Noone took this role in our team. As the result, we've spent almost whole Saturday developing and polishing algorithm for enemy movement and around half an hour brainstorming and then voting for the game title, while our game used untextured, placeholder cubes and spheres as models until the very end :)
Conclusion: It's not enough to know how to code. It's also important to decide WHAT to code so that best possible result can be achieved with limited time and resources.
But the Global Game Jam as a whole is not a contest (despite our site actually was one, with PlayStation 4 for each team member as first prize) but just a fun, creative event. Despite all the problem we had I think it was fun. I had yet another opportunity to use Unity, which is a great technology. I realized I can handle Git pretty well, despite I don't feel like an expert knowing about "rebase" and such advanced stuff. I realized I still remember how to use the so much unintuitive inteface of Blender, which I learned many years ago to use in my master thesis. I could play many interesting games created on this jam, like my favorite: Witch Rite (it took 3rd place) or the one that won the contest: Acolytes: Ritual of Ascension. And finally, I've met many interesting people who do all sorts of crazy stuff, from running a company that produces medical software and hardware, to visiting escepe rooms and practicing celtic dances :)