Programming, graphics, games, media, C++, Windows, Internet and more...
# Two Shader Compilers of Direct3D 12
Mon
23
Dec 2019
If we write a game or other graphics application using DX12, we also need to write some shaders. We author these in high-level language called HLSL and compile them before passing to the DirectX API while creating pipeline state objects (ID3D12Device::CreateGraphicsPipelineState). There are currently two shader compilers available, both from Microsoft, each outputting different binary format:
Which one to choose? The new compiler, called DirectX Shader Compiler, is more modern, based on LLVM/Clang, and open source. We must use it if we want to use Shader Model 6 or above. On the other hand, shaders compiled with it require relatively recent version of Windows and graphics drivers installed, so they won’t work on systems not updated for years.
Shaders can be compiled offline using a command-line program (standalone executable compiler) and then bundled with your program in compiled binary form. That’s probably the best way to go for release version, but for development and debugging purposes it’s easier if we can change shader source just as we change the source of CPU code, easily rebuild or run, or even reload changed shader while the app is running. For this, it’s convenient to integrate shader compiler as part of your program, which is possible through a compiler API.
This gives us 4 different ways of compiling shaders. This article is a quick tutorial for all of them.
1. Old Compiler - Offline
The standalone executable of the old compiler is called “fxc.exe”. You can find it bundled with Windows SDK, which is installed together with Visual Studio. For example, in my system I located it in this path: “c:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\fxc.exe”.
To compile a shader from HLSL source to the old binary format, issue a command like this:
fxc.exe /T ps_5_0 /E main PS.hlsl /Fo PS.bin
/T is target profileps_5_0 means pixel shader with Shader Model 5.0/E is the entry point - the name of the main shader function, “main” in my casePS.hlsl is the text file with shader source/Fo is binary output file to be written
There are many more command line parameters supported for this tool. You can display help about them by passing /? parameter. Using appropriate parameters you can change optimization level, other compilation settings, provide additional #include directories, #define macros, preview intermediate data (preprocessed source, compiled assembly), or even disassemble existing binary file.
2. Old compiler - API
To use the old compiler as a library in your C++ program:
#include <d3dcompiler.h>D3DCompileFromFileExample:
CComPtr<ID3DBlob> code, errorMsgs;
HRESULT hr = D3DCompileFromFile(
L"PS.hlsl", // pFileName
nullptr, // pDefines
nullptr, // pInclude
"main", // pEntrypoint
"PS_5_0", // pTarget
0, // Flags1, can be e.g. D3DCOMPILE_DEBUG, D3DCOMPILE_SKIP_OPTIMIZATION
0, // Flags2
&code, // ppCode
&errorMsgs); // ppErrorMsgs
if(FAILED(hr))
{
if(errorMsgs)
{
wprintf(L"Compilation failed with errors:\n%hs\n",
(const char*)errorMsgs->GetBufferPointer());
}
// Handle compilation error...
}
D3D12_GRAPHICS_PIPELINE_STATE_DESC psoDesc = {};
// (...)
psoDesc.PS.BytecodeLength = code->GetBufferSize();
psoDesc.PS.pShaderBytecode = code->GetBufferPointer();
CComPtr<ID3D12PipelineState> pso;
hr = device->CreateGraphicsPipelineState(&psoDesc, IID_PPV_ARGS(&pso));
First parameter is the path to the file that contains HLSL source. If you want to load the source in some other way, there is also a function that takes a buffer in memory: D3DCompile. Second parameter (optional) can specify preprocessor macros to be #define-d during compilation. Third parameter (optional) can point to your own implementation of ID3DInclude interface that would provide additional files requested via #include. Entry point and target platforms is a string just like in command-line compiler. Other options that have their command line parameters (e.g. /Zi, /Od) can be specified as bit flags.
Two objects returned from this function are just buffers of binary data. ID3DBlob is a simple interface that you can query for its size and pointer to its data. In case of a successful compilation, ppCode output parameter returns buffer with compiled shader binary. You should pass its data to ID3D12PipelineState creation. After successful creation, the blob can be Release-d. The second buffer ppErrorMsgs contains a null-terminated string with error messages generated during compilation. It can be useful even if the compilation succeeded, as it then contains warnings.
Update: "d3dcompiler_47.dll" file is needed. Typically some version of it is available on the machine, but generally you still want to redistribute the exact version you're using from the Win10 SDK. Otherwise you could end up compiling with an older or newer version on an end-user's machine.
3. New Compiler - Offline
Using the new compiler in its standalone form is very similar to the old one. The executable is called “dxc.exe” and it’s also bundled with Windows SDK, in the same directory. Documentation of command line syntax mentions parameters starting with "-", but old "/" also seems to work. To compile the same shader using Shader Model 6.0 issue following command, which looks almost the same as for "fxc.exe":
dxc.exe -T ps_6_0 -E main PS.hlsl -Fo PS.bin
Despite using a new binary format (called “DXIL”, based on LLVM IR), you can load it and pass it to D3D12 PSO creation the same way as before. There is a tricky issue though. You need to attach file “dxil.dll” to your program. Otherwise, the PSO creation will fail! You can find this file in Windows SDK path like: “c:\Program Files (x86)\Windows Kits\10\Redist\D3D\x64\dxil.dll”. Just copy it to the directory with target EXE of your project or the one that you use as working directory.
4. New Compiler - API
The new compiler can also be used programatically as a library, but its usage is a bit more difficult. Just as with any C++ library, start with:
#include <dxcapi.h>IDxcCompilerThis time though you need to bundle additional DLL to your program (next to “dxil.dll” mentioned above): “dxcompiler.dll”, to be found in the same “Redist\D3D\x64” directory. There is more code needed to perform the compilation. First create IDxcLibrary and IDxcCompiler objects. They can stay alive for the whole lifetime of your application or as long as you need to compile more shaders. Then for each shader, load it from a file (or any source of your choice) to a blob, call Compile method, and inspect its result, whether it’s an error + a blob with error messages, or a success + a blob with compiled shader binary.
CComPtr<IDxcLibrary> library;
HRESULT hr = DxcCreateInstance(CLSID_DxcLibrary, IID_PPV_ARGS(&library));
//if(FAILED(hr)) Handle error...
CComPtr<IDxcCompiler> compiler;
hr = DxcCreateInstance(CLSID_DxcCompiler, IID_PPV_ARGS(&compiler));
//if(FAILED(hr)) Handle error...
uint32_t codePage = CP_UTF8;
CComPtr<IDxcBlobEncoding> sourceBlob;
hr = library->CreateBlobFromFile(L"PS.hlsl", &codePage, &sourceBlob);
//if(FAILED(hr)) Handle file loading error...
CComPtr<IDxcOperationResult> result;
hr = compiler->Compile(
sourceBlob, // pSource
L"PS.hlsl", // pSourceName
L"main", // pEntryPoint
L"PS_6_0", // pTargetProfile
NULL, 0, // pArguments, argCount
NULL, 0, // pDefines, defineCount
NULL, // pIncludeHandler
&result); // ppResult
if(SUCCEEDED(hr))
result->GetStatus(&hr);
if(FAILED(hr))
{
if(result)
{
CComPtr<IDxcBlobEncoding> errorsBlob;
hr = result->GetErrorBuffer(&errorsBlob);
if(SUCCEEDED(hr) && errorsBlob)
{
wprintf(L"Compilation failed with errors:\n%hs\n",
(const char*)errorsBlob->GetBufferPointer());
}
}
// Handle compilation error...
}
CComPtr<IDxcBlob> code;
result->GetResult(&code);
D3D12_GRAPHICS_PIPELINE_STATE_DESC psoDesc = {};
// (...)
psoDesc.PS.BytecodeLength = code->GetBufferSize();
psoDesc.PS.pShaderBytecode = code->GetBufferPointer();
CComPtr<ID3D12PipelineState> pso;
hr = device->CreateGraphicsPipelineState(&psoDesc, IID_PPV_ARGS(&pso));
Compilation function also takes strings with entry point and target profile, but in Unicode format this time. The way to pass additional flags also changed. Instead of using bit flags, parameter pArguments and argCount take an array of strings that can specify additional parameters same as you would pass to the command-line compiler, e.g. L"-Zi" to attach debug information or L"-Od" to disable optimizations.
Update 2020-01-05: Thanks @MyNameIsMJP for your feedback!
Comments | #rendering #directx Share
# Xiaomi Smart Band - a Very Good Purcharse
Fri
20
Dec 2019
Despite being a programmer, I'm quite conservative with technological novelties. For example, I never owned a tablet and never felt a need to have one. I was also aware there are smart watches on the market, but an idea of spending 300 EUR for another device that I would then need to charge every day seemed like too much for me. Neither I was interested in those smart bands that claim to monitor your pulse, sleep, and count your steps.
What made me revisit those types of devices was a real need I felt many times recently. Sometimes I was attending a conference, sitting on a talk with my smartphone switched to quiet mode. I was repeatedly pulling it out to check the time or to see if anyone tried to call me or sent me a message. Those are the situations where it's important not to miss the talk I want to see and to catch up with my friends, while having my phone ringing would be undesirable. Another time I was on a party in a club or concert where the music was so loud I couldn't hear or feel my phone ringing, while that's also the time when I repeatedly check the clock not to miss the show of my favourite DJ while trying to hang out with my friends. Then I thought: maybe a smart watch could provide those two simple things: show the current time and display notifications from text messages, Messenger etc., notifying about them using vibration?
After some research I found out that smart bands do exactly that if I disregard all sport-related features. I chose Xiaomi Mi Smart Band 4, but devices from other manufacturers would probably provide the similar experience. It surprised me it costs only 140 PLN (33 EUR). After charging the battery, installing special "Mi Fit" app on my Android phone and pairing the two devices using Bluetooth, I could configure my smart band to change wallpaper etc. (BTW the default ones are terrible, but their format has been reverse engineered, so there are many user-created watchfaces available to download on the Internet.)
This device has only 512 KB of RAM and 120x240 pixel screen, but even with those parameters looking like some Atari computer from 30 years ago it provides many useful features. First and foremost, it shows current time and date - for 5 seconds after activated using touch screen. It also vibrates when there is an incoming call or a new message. You can configure which of the apps installed on the smartphone display their notifications also on the band, but these can be basically the same as the notifications on the phone, so any messaging app will work - whether Messenger, WhatsApp, Signal, Tinder, or standard text messages. Sender and text can be seen on the band, but responding requires pulling out the phone. Additional features I like are showing current weather and the weather forecast for the following days for current location, timer, and alarms, which can wake you up using vibration - useful not to wake up other people sleeping in the same room. The biggest surprise for me was battery life. 20 days declared by the manufacturers seemed unrealistic, but after I charged it for the first time on November 20th, it worked until... yesterday, which gives 29 days. One disadvantage I can see is that now I must have Bluetooth enabled on my phone all the time, which drains its battery faster, but I charge the phone every day anyway.
This article is not sponsored. I wrote it from my initiative, just to share my experiences with this type of device. If you consider it useful to have current time and incoming messages available on your wristband without a need to pull out your phone, a smart band like Xiaomi Mi Smart Band 4 is a good choice.
Update 2020-07-12: Regarding notifications, see also my post: How to Disable Notification Sound in Messenger for Android?