Programming, graphics, games, media, C++, Windows, Internet and more...
All blog entries, ordered from most recent. Entry count: 1198.
Pages: 
1 ... 62 63 64 65 66 ... 150 
# My Way of Implementing Streams #2
Sat
15
Aug 2009
In my previous post My Way of Implementing Streams #1 I've revealed what's my attitude towards stream class hierarchy. I want to explore this subject further by looking into details of my code. Let's start with writing and reading a single value to/from a stream. I know that processing bigger chunk of data with single call is more optimal, but sometimes we want to save a single number, boolean value, vector or whatever. We traditionally do it by passing address of the value and its size, just like that:
fwrite(&value, sizeof(value), 1, file);
But in C++ we can automate this by defining a template method, which I like to call "WriteEx":
template <typename T>
void WriteEx(const T &x)
{
return Write(&x, sizeof(x));
}
template <typename T>
void ReadEx(T *x)
{
MustRead(x, sizeof(*x));
}
Now I can read and write single values from/to my streams much easier:
stream.WriteEx(value);
Character strings are special kinds of values because they have variable size. Traditional C approach is to keep strings NULL-terminated, but I don't like this. I prefer to remember string length together with characters because: 1. I can do stream read and write with just two calls (one for length and one for data) instead of reading/writing byte after byte until I meet '\0', 2. I know how many bytes to allocate for character buffer when reading a string, 3. string can contain any characters including 0. On the other hand, I have to decide how many bytes do I want to use as string length, thus limiting maximum string length to 255 B (1 B), 64 KB (2 B) or 4 GB (4 B). To leave this choice to the user, I've defined many functions for reading and writing string to my streams:
// Write string preceded by length as 1 B void WriteString1(const string &s); // Write string preceded by length as 2 B void WriteString2(const string &s); // Write string preceded by length as 4 B void WriteString4(const string &s); // Write only string content void WriteStringF(const string &s); // Read string preceded by length as 1 B void ReadString1(string *s); // Read string preceded by length as 2 B void ReadString2(string *s); // Read string preceded by length as 4 B void ReadString4(string *s); // Read only string content void ReadStringF(string *s, size_t NumChars); // This one does not need a comment :) void ReadStringToEnd(string *s);
Another thing I want to mention is the concept of skipping bytes while reading from a stream. Sometimes we want to ignore some bytes, so I've defined Skip method for this purpose.
virtual size_t Skip(size_t MaxLength); void MustSkip(size_t Length);
Interesting here is that I provide default implementation in my both base stream classes. Default implementation in Stream class just reads given number of bytes to an internal buffer and ignores it:
size_t Stream::Skip(size_t MaxLength)
{
if (MaxLength == 0) return 0;
char Buf[BUFFER_SIZE];
uint BlockSize, ReadSize, Sum = 0;
while (MaxLength > 0)
{
BlockSize = std::min(MaxLength, BUFFER_SIZE);
ReadSize = Read(Buf, BlockSize);
Sum += ReadSize;
MaxLength -= ReadSize;
if (ReadSize < BlockSize)
break;
}
return Sum;
}
In the SeekableStream class more optimal algorithm can be used, which just sets cursor position. Derived classes of specific types of streams are still free to reimplement Skip method to provide even better versions.
size_t SeekableStream::Skip(size_t MaxLength)
{
uint Pos = GetPos();
uint Size = GetSize();
uint BytesToSkip = std::min(MaxLength, Size - Pos);
SetPosFromCurrent((int)BytesToSkip);
return BytesToSkip;
}
Last thing I want to tell about today are some utility methods and functions which I've created just for convenience. I think their names are self-explanatory :)
class Stream
{
...
size_t CopyFrom(Stream *src, size_t Size);
void MustCopyFrom(Stream *src, size_t Size);
void CopyFromToEnd(Stream *src);
};
void SaveStringToFile(const std::string &FileName, const std::string &Data);
void LoadStringFromFile(const std::string &FileName, std::string *Data);
void SaveDataToFile(const std::string &FileName, const void *Data, size_t NumBytes);
Next time I will show specific classes of my streams with some examples of how do I use them.
# My Way of Implementing Streams #1
Thu
13
Aug 2009
Today I want to write a bit about a part of my library containing classes that represent binary data streams. My concept of streams is based on Delphi approach and maybe also Java. I use it for many years and I think it's very convenient. General assumtions are: I use object-oriented approach with base, abstract Stream class and virtual methods. I report errors using C++ exceptions. Of course they both cause some performance overhead, but flexibility and good error control has higher priority for me. Reading and writing disk files is orders of magnitude slower than CPU computations anyway :)
Streams in C++ standard library have separate classes for input stream (stream that can be read) and output stream (stream that can be written). I don't like this idea. First, disk files can be opened for both reading and writing, which forces basic_iostream class to use multiple inheritance and basic_ios to be inherited virtually. Second, I rather see opening a file for writing and/or reading as a mode, wchich is more suitable for constructor parameter than for separate classes. I prefer different division instead. I distinguish simple streams - these that can be only read and/or written and "seekable" streams - these that can also tell their data size as well as get and set current cursor position.
There are still two issues related to data streams that have to be considered. First is an end of stream concept. Standard library of C, Java and many other languages handle this in a way that looks totally weird for me. You can know about reaching end of file only if you have already tried to read past the end and that operation failed. Another strange thing is reading single byte (getc function) but returing type int instead of char, where value EOF == -1 means end of file. Maybe that's why some (beginner) programmers think that there is a magical "end of file" character standing at the end of every file, while the truth is that file has just its remembered size in bytes and each byte can have any of 256 possible values.
My approach for handling end of file is to implement bool End() method which returns true if the cursor is at the end of stream so that no more bytes can be read (cursor position == data size). Of course I have standard Read method which tries to read as many bytes as possible up to given buffer size and returns number of bytes read. It's convenient when you want to process a big file without loading it all to memory, using a buffer of constant size.
But I believe that more often you want to read a single value (like a number or a header structure) and you expect this read operation to succeed. If you do I/O with standard C, C++ or WinAPI functions, admit to yourself now whether you check for success of any read operation as well as number of bytes actually read? If you don't, then analyze what happens if you are already at the end of file and you do the following code:
unsigned elemCount; fread(&elemCount, sizeof(elemCount), 1, myFile); // What's the value of elemCount now and what will happen next?! Vector3 *vectors = new Vector3[elemCount]; fread(vectors, sizeof(Vector3), elemCount, myFile);
That's why I also implement MustRead method, which throws an exception if expected number of bytes couldn't be read. I also expect every write operation to fully succeed by writing all bytes given or to throw the exception if it couldn't be done. Thanks to that I can be sure that I will always know about any failure in reading or writing any single value.
Second thing that has to be stated is whether any read/write operation can fail (or just not read all expected bytes) "for now", but succeed later. For example that's the case for network sockes, where write buffer can be currently full and read buffer can be empty so you have to wait for a while to be able to write/read more bytes. I assume that my streams cannot work this way, so I can't implement socket as a class derived from my Stream. It simplifies my streams because now I know that if I can't read more bytes, I've just reached end of data.
That's it for now. This entry is long enought :) I'll continue this streams topic next time. Let's say the interface of my base stream classes looks like this:
class Stream
{
public:
virtual ~Stream() { }
virtual void Write(const void *Data, size_t Size);
virtual void Flush() { }
virtual size_t Read(void *Data, size_t MaxLength);
virtual void MustRead(void *Data, size_t Length);
virtual bool End();
...
};
class SeekableStream : public Stream
{
public:
virtual size_t GetSize();
virtual void SetSize(size_t Size);
virtual int GetPos();
virtual void SetPos(int pos);
virtual void SetPosFromCurrent(int pos);
virtual void SetPosFromEnd(int pos);
...
};
# Magic Numbers in Visual C++
Mon
10
Aug 2009
Uninitialized variables in C++ language are not filled automatically. They can contain any random data. It's as good as no range checking in access to arrays, because it doesn't waste time out of programmer's control. But on the other hand this rule makes us programmers responsible for thinking about when, where and how do we ensure that we don't use variables before their initialization or index arrays out of their range.
Visual C++ helps us with that. While in Release compilation, variables and fields are really uninitialized and contain random data, but under Debug configuration they are filled with special magic numbers. If we debug the program, hexadecimal codes of these numbers can tell us about the kind of error we made in the code. Just press "Hex" button on Debug toolbar to see all numbers as hexadecimal. Here are some of magic numbers used by Visual C++:
Interesting thing is that compiler doesn't warn about uninitialized fields in classes, despite it knows about uninitialized local variables and shows "Run-Time Check Failure #3 - The variable 'v' is being used without being initialized.". So whenever you have a bug which appears only in Release configuration and not in Debug, good chances are that you've forgotten to initialize a field in a class.
By the way, some programmers think that debugging is possible only in Debug configuration (as its name says). You acually can debug also in Release mode, although step by step execution may sometimes work strange due to optimizations like inlining, some watches may show invalid values and sometimes looking into objects doesn't work (probably bacause of optimized way "this" pointer is passed).
Comments | #visual studio #c++ Share
# Dialog Layout Manager in MFC
Sat
08
Aug 2009
Sometimes I write some tools using C++ and MFC. In the Linux world it is common that GUI windows are resizeable. In Windows its not the case, but sometimes it would be nice to be able to resize a dialog window to see more information, like more rows and colums in a list. After repositioning and resizing controls in a window with my custom code I've decided to automate this task.
There are many possible approaches to this problem. WinAPI (and thus MFC) does not provide by itself any solution to automatically align controls inside a resizeable window. Each control has just its fixed rectangle (left, top, width, height) inside parent window. Delphi VCL uses Align property to snap selected controls (like Panel containing child controls) to left, right, top or bottom edge of the window. Qt encourages to design all windows with Layouts. For example, Vertical Layout splits the window into rows and automatically adjusts controls inside, one under the other.
But the solution of my choice is the one based on .NET. Controls in Windows Forms have a property called Anchor so they can be anchored to any of four possible window edges: left, top, right and bottom. If a controls is anchored only to left and top edges, it just doesn't change its position or size. If the control is anchored to right and bottom edges (for example: a button), it changes its position as window is resized so it preserves its distance to right and bottom edge of the window. If the control is anchored to all four possible window edges, it is resized to preserve distance to all window edges same as designed (for example: a list occupying central part of the window).
I've written a class which I called DialogLayoutManager. It's very easy to use and automates control resizing and repositioning inside an MFC window. All you need to do at the beginning is to create an object of this class, register your controls with selected anchors and call Save method:
m_LayoutManager->SetControl(GetDlgItem(ID_BTN_CANCEL), DialogLayoutManager::ANCHOR_RIGHT | DialogLayoutManager::ANCHOR_BOTTOM); m_LayoutManager->SetControl(GetDlgItem(ID_BTN_OK), DialogLayoutManager::ANCHOR_RIGHT | DialogLayoutManager::ANCHOR_BOTTOM); m_LayoutManager->Save(this);
Layout manager remembers positions and sizes of registered controls together with starting window size. Now all you need to do when the window is resized is to call Restore method. Layout manager will adjust registered controls according to new window size and specified anchors. For example, two buttons showed above will stay in bottom-right corner of the window.
void CDialog01::OnSize(UINT nType, int cx, int cy)
{
...
if (LayoutManager && LayoutManager->IsSaved())
m_LayoutManager->Restore(this);
}
Here is the code of my DialogLayoutManager class and usage example: DialogLayoutManager.cpp. It's easy to translate this code to pure WinAPI.
Comments | #gui #mfc #winapi Share
# How Do People Write Bad Code
Wed
05
Aug 2009
Recently yarpen posted some thoughts about source code scalability on his blog. His blog entries are always very valuable and this time he touched broader subject which is connected to software antipatterns.
I want to expand this subject further and think a bit about how is such bad code created (we all agree that non-scalable code is bad, right? :) In my personal opinion it's generally caused by the lack of thinking. For example, if a coder have to code something (or, how some people would like to say: to solve a problem), he should first consider what possible solutions are. This may be choice of a technology, programming languages and libraries, some design patterns, algorithms, data structures or even names for some objects. If he doesn't think about it, he just selects the first option that came to his mind without considering alternatives and thus risks that he didn't make the best possible decision.
Next, the coder should design his solution. Designing is all about thinking before doing. If he skips this step, he may meet during his coding a situation when sees he came the wrong way and now he must refactor or even delete big part of his code and then just... rethink it :) I like the saying It's better to think twice before you start coding than to code twice before you start thinking.
:)
The same applies to all changes made to an existing code. If somebody have to change something in a program, he'd better review the code carefully to discover and understand its architecture. If he just starts making changes to the code, he risks that he will break some hidden assumptions in some aspects of the code (like variable values, objects lifetime, order of function calls etc.) and thus create bugs.
The reason behind writing poor code may be rush. If a coder wants to code something as quick and simply as possible (or his boss tells him to do it this way), he will do ugly hacks and never refactor anything, even if some refactoring is necessary at this point. Of course this it to bad code and will backfire in the future, when the code will have to be changed and it will turn out to be totally unscalable. So we obviously see that such attitude is just the lack of thinking.
But this problem may also be something deeper. I can see that some coders just always write bad, unscalable code by their nature. I don't want to offend anyone, but I think maybe that's the crucial difference between junior and more experienced programmers. To work as a programmer, everyone needs to "speak fluently" his programming language to freely express his thoughts using it, but experienced programmers are also able to see a bigger picture - some invisible assumptions and more abstract concepts behind the code. They see words like "handle", "manager", "alignment", "serialization", "callback" and hundreds of other terms (anyone thought about creating coder's dictionary with words like these?) and just knows what hides behind them.
Three days after yarpen's entry there appeared a blog entry about code documentation on EntBlog. It describes two main types of documents - API documentation and technical articles. I fully agree with this text. I value documentation very much and I think that good documentation is another piece of this big puzzle.
They say that there are two types of people: these who make backups and these who will make it
:) I think it's the same with writing good code and documentation. It's just the matter of thinking forward.
Comments | #software engineering #philosophy Share
# Backup Script in PHP
Sun
02
Aug 2009
One of my personal security procedures is daily backup of the code I'm working on to my FTP server. This process requires packing files to an archive and upload the archive to the FTP server. Most annoying part when doing it manually was always selecting aproppriate files, without all these big temporary files and directories like Debug, Release, .ncb, .suo, .user etc.
Today in the morning I've decided to automate this backup task with a script. As I like to use PHP as shell scripting language, here is my backup script in PHP: Backup.php_. It was not a suprise for me to see that PHP already have all the functionality I need, including building ZIP archive and FTP client. Some comments to my code:
To run the script, you must have PHP installed. Then you can just enter something like this into command line:
php Backup.php D:\MyGreatProject
Or you can create BAT file to support backup of your particular project:
@php D:\Backup.php D:\MyGreatProject @pause
After the script have been started, you have to enter "backup name". It is the name of the ZIP file that will be uploaded to FTP. Script recursively searches for files and subdirectories in the given project directory, filters them using functions FilterDir and FilterFile, packs them to a temporary archive file and then uploads the archive to the FTP server. All remaining parameters (temporary archive path, FTP hostname, login, password and remote directory) are hardcoded as constants.
When it comes to the script code, one of the issues was to ask user about the backup name using system console. PHP has an extension called Readline, but unfortunately it doesn't work on Windows. The solution was to use fgets function with STDIN constant:
print("Enter backup name (default: \"$sSuggestedBackupName\"): ");
$sBackupName = trim(fgets(STDIN, 1024));
if (!$sBackupName)
$sBackupName = $sSuggestedBackupName;
When it comes to building ZIP archive and handling FTP connection, PHP also has extensions for that and they seem to just work in my PHP without any problems. Zip extension exposes class ZipArchive and FTP extension exposes functions ftp_, like ftp_connect or ftp_put. All is documented in the official PHP documentation.
Comments | #scripts #php #tools Share
# Sunrise 2009 in Kołobrzeg
Tue
28
Jul 2009
I didn't post anything on my blog in recent days because I've been offline and away from keyboard on my holiday in Kołobrzeg. It's a nice city at the seaside, but I haven't go there to spend time on the beach taking sunbath :) I've been on Sunrise Festival - a big, 3-day party with trance/house music. I've also made about 800 photos with my digital camera and cellphone. You can see some of them in my new gallery: Sunrise 2009.
Comments | #music #life #events Share
# Three Ways to Calculate Mean and Variance
Mon
20
Jul 2009
There is a free e-book on DSP (Digital Signal Processing) called The Scientist and Engineer's Guide to Digital Signal Processing (by Steven W. Smith, Ph.D.). As I have a holiday now, I've started reading it and I've found some interesting information even in introductory chapters. For example, there are three algorithms to calculate mean and variance. Let's say we have some data in a vector of bytes and we want to calculate its statistics.
std::vector<unsigned char> Bytes; // Fill Bytes... uint N = Bytes.size();
Traditional algorithm looks like this:
float Mean = 0.f, Variance = 0.f; for (uint i = 0; i < N; i++) Mean += (float)Bytes[i]; Mean /= (float)N; for (uint i = 0; i < N; i++) Variance += ((float)Bytes[i] - Mean) * ((float)Bytes[i] - Mean); Variance /= (float)(N-1);
Now the incremental one, which can return current mean and variance at any time while you can also post next piece of data. All we need to implement it is to keep track of current sample number, sum of samples and sum of squared samples. I've created template class for that, which can be parametrized by float, double, int or other numeric types.
template <typename T>
class IncrementalMeanAndVarianceCalc
{
public:
IncrementalMeanAndVarianceCalc() : m_Sum(T()), m_SqSum(T()), m_Count(0) { }
void Clear() { m_Sum = m_SqSum = T(); m_Count = 0; }
void Add(const T &v)
{
m_Sum += v;
m_SqSum += v*v;
m_Count++;
}
void Add(const T *values, uint valCount)
{
for (uint i = 0; i < valCount; i++)
{
m_Sum += values[i];
m_SqSum += values[i]*values[i];
}
m_Count += valCount;
}
bool IsEmpty() { return m_Count == 0; }
uint GetCount() { return m_Count; }
T GetSum() { return m_Sum; }
T GetSqSum() { return m_SqSum; }
T GetMean()
{
assert(!IsEmpty());
return m_Sum / m_Count;
}
T GetVariance(bool varianceBiased = true)
{
if (varianceBiased)
return (m_SqSum - m_Sum*m_Sum/m_Count) / (m_Count-1);
else
return (m_SqSum - m_Sum*m_Sum/m_Count) / m_Count;
}
void GetMeanAndVariance(T &outMean, T &outVariance, bool varianceBiased = true)
{
assert(!IsEmpty());
outMean = m_Sum / m_Count;
if (varianceBiased)
outVariance = (m_SqSum - m_Sum*m_Sum/m_Count) / (m_Count - 1);
else
outVariance = (m_SqSum - m_Sum*m_Sum/m_Count) / m_Count;
}
private:
T m_Sum, m_SqSum;
uint m_Count;
};
Finally, there is an algorithm to calculate mean and variance from histogram. It is very efficient method. If we have only 256 possible values for each sample, we don't have to do binning and calculating histogram is very simple:
uint Histogram[256] = { 0 };
for (uint i = 0; i < N; i++)
Histogram[Bytes[i]]++;
Now, the mean and variance can be calculated this way:
float Mean = 0.f, Variance = 0.f; for (uint i = 0; i < 256; i++) Mean += (float)i * Histogram[i]; Mean /= (float)N; for (uint i = 0; i < 256; i++) Variance += (i - Mean)*(i - Mean) * Histogram[i]; Variance /= N - 1;