Programming, graphics, games, media, C++, Windows, Internet and more...
Sat
23
May 2026
Recently, I dove deep into floating-point numbers and their behavior. Somehow, this topic haunts me in my programming practice since I created Floating-Point Formats Cheatsheet back in 2013 and also released a comprehensive article The Secrets of Floating-Point Numbers in 2024.
This time, I would like to focus on one specific question:
What is the result of floor(-1.175493930432748e-38) ?
Note: Hexadecimal value of our input number is 0x807FFFFD.
To recap, floor, ceil, trunc, round are functions available in the standard library of C, C++, as well as shading languages: HLSL and GLSL. Each of them transforms a floating-point number into an integral floating-point value, but using different rounding rules.
Note this is not about a conversion from float to int. The result of these functions is still a float, just having only integral part. When the input is already integral, the value is returned as-is. Otherwise, it gets "snapped" to the nearest integer in a specific direction:
floor - rounding "down" i.e., towards -infinity.ceil - rounding "up" i.e., towards +infinity.trunc - rounding towards zero, which we can also explain as truncating the fractional part.round - rounding to the nearest integer up or down, depending on which one is closer.Examples:
Note: IEEE 754 floating-point numbers distinguish between positive and negative zero, so some results below are -0.0 rather than 0.0 to visualize this distinction.
floor( 5.7) = 5.0 ceil( 5.7) = 6.0 trunc( 5.7) = 5.0 round( 5.7) = 6.0 floor( 0.2) = 0.0 ceil( 0.2) = 1.0 trunc( 0.2) = 0.0 round( 0.2) = 0.0 floor(-0.2) = -1.0 ceil(-0.2) = -0.0 trunc(-0.2) = -0.0 round(-0.2) = -0.0 floor(-5.7) = -6.0 ceil(-5.7) = -5.0 trunc(-5.7) = -5.0 round(-5.7) = -6.0
When talking about round, there is also a question what happens when we are exactly halfway, like round(2.5). Various programming languages define it differently:
round(2.5) = 3.0, round(-3.5) = -4.0.round(2.5) = 2.0, round(-3.5) = -4.0.round implementation-dependent, while also offering function roundEven that rounds towards nearest even.Knowing all this, we can answer our main question:
According to mathematical rules, floor(-1.175493930432748e-38) = -1.0, because the number is between -1 and 0.
However, those of you who know more about the structure of floating-point numbers may notice that our input value is a subnormal. Subnormal numbers, also called denormalized numbers or denormals, are values so small (so close to 0) that they use a special representation where the implicit leading 1 bit is no longer assumed. They have exponent = 0. The minimum positive normalized value representable by 32-bit floats is 1.18 * 10^-38, while the minimum value representable as denormalized is 1.4 * 10^-45, so our number falls in that range.
We wouldn't need to care about denormals if not for the fact that:
This is the problem I stumbled upon recently. In most cases, it doesn't matter. For example, when rendering graphics, the difference between such a small number and 0 would produce an indistinguishable difference in results. After applying functions such as floor and ceil, however, the difference is significant:
If the platform preserves denormals: floor(-1.175493930432748e-38) = -1.0 ceil(-1.175493930432748e-38) = -0.0 floor( 1.175493930432748e-38) = 0.0 ceil( 1.175493930432748e-38) = 1.0 If the platform flushes denormals to 0: floor(-1.175493930432748e-38) = -0.0 ceil(-1.175493930432748e-38) = -0.0 floor( 1.175493930432748e-38) = 0.0 ceil( 1.175493930432748e-38) = 0.0
The behavior of a specific platform may depend on many factors, such as flags used during compilation of our source code, as well as some floating-point modes controlled in runtime. It may be an unexpected source of nondeterminism between CPU and GPU, as well as between GPU vendors.
I've performed a few tests. Here are my results:
floor and ceil. I've tested the following options, with no change in the results:
/fp:precise, /fp:strict, /fp:fast/Oi/arch:SSE2, /arch:AVX2-Gis (Force IEEE strictness)-denorm preserve-denorm preserve makes it preserving denormals.-denorm preserve makes it preserving denormals.This is not the first time we can see Nvidia taking shortcuts to achieve maximum performance of their GPUs 😉 We could see another example in the article "Mipmap selection in too much detail" by Pema Malling.
Note: Architectures may support two related modes: flushing denormal results to zero and treating denormal inputs as zero. The behavior observed here suggests the latter.
UPDATE: As Pete Cawley commented on X, DirectX Specification, chapter 3.1.3.2 "Complete Listing of Deviations or Additional Requirements vs. IEEE-754" actually requires GPUs to flush denorms on input and output of any floating point operation.
If, for any reason, you need an implementation of these functions that behaves consistently, deterministically, and preserving denormals across CPUs and all GPUs, you can use the following HLSL code implementing custom floor and ceil using simple bit tricks:
float DeterministicFloor(float x)
{
if ((asuint(x) & 0xFF800000u) == 0x80000000 && // sign = 1, exponent = 0
(asuint(x) & 0x007FFFFFu) != 0) // mantissa != 0
{
return -1.0f;
}
return floor(x);
}
float DeterministicCeil(float x)
{
if ((asuint(x) & 0xFF800000u) == 0 && // sign = 0, exponent = 0
(asuint(x) & 0x007FFFFFu) != 0) // mantissa != 0
{
return 1.0f;
}
return ceil(x);
}