Tag: software engineering

Entries for tag "software engineering", ordered from most recent. Entry count: 33.

Uwaga! Informacje na tej stronie mają ponad 5 lat. Nadal je udostępniam, ale prawdopodobnie nie odzwierciedlają one mojej aktualnej wiedzy ani przekonań.

Pages: > 1 2 3 4 5 >

# Data-Oriented Design - Links and Thoughts

Jan 2011

In April 2008 I've written an essay "Fanatyzm obiektowy" (in Polish, it means "Object-Oriented Fanaticism"). I've always believed there is something wrong with object-oriented programming, that it simply doesn't meet its own objectives and so following it blindly as an ideology not only a programming language mechanics has many pitfalls. Now I'm glad that recently a concept of "Data-Oriented Design" (DOD) emerged and gained popularity among game developers. Here is my try to aggregate all important information on this subject that can be found on the Internet:


Blog entries:


If you know any other good readings on this subject, please leave a comment. I'll update my list.

As far as I can see, focusing more on data instead of objects gives a number of benefits for the code:

Of course DOD doesn't exist in the void. It's related to many other concepts and you can find many good sources of knowledge about each one. Some of them are:

Comments | #software engineering #c++ Share

# Different Ways of Processing Data

Dec 2010

Much is being said about UX these days, while not so much about the art and science of designing good API so libraries can communicate successfully with their users, that is other programmers. I'm interested in the latter for some time and today I'd like to explore the topic of how some data, whether objects or a raw sequence of bytes, can be processed, loaded and saved. I will clarify what I mean in just a moment. I believe the ways to do it can be grouped into several categories, from simplest but most limited way to the most flexible, efficient but difficult to use.

1. The simplest possible interface for loading (or saving) some data is to pass a string with path to file. That's the way we load DLL libraries in WinAPI. Unfortunately it limits the programmer to load object only from physical files, not from other places like memory pointer, where source data could be placed e.g. after decompression or downloading from network.

HMODULE WINAPI LoadLibrary(__in LPCTSTR lpFileName);

2. Solution to this problem is an API that allows loading data from either file or memory. Some example can be texture loading in D3DX (extension to DirectX), where separate functions are available that take either path to a disk file (LPCTSTR pSrcFile), pointer to a buffer in memory (LPCVOID pSrcData, UINT SrcDataSize) or Windows resource identifier (HMODULE hSrcModule, LPCTSTR pSrcResource).

HRESULT D3DXCreateTextureFromFile(
  __in LPDIRECT3DDEVICE9 pDevice,
  __in LPCTSTR pSrcFile,
  __out LPDIRECT3DTEXTURE9 *ppTexture);
HRESULT D3DXCreateTextureFromFileInMemory(
  __in LPDIRECT3DDEVICE9 pDevice,
  __in LPCVOID pSrcData,
  __in UINT SrcDataSize,
  __out LPDIRECT3DTEXTURE9 *ppTexture);
HRESULT D3DXCreateTextureFromResource(
  __in LPDIRECT3DDEVICE9 pDevice,
  __in HMODULE hSrcModule,
  __in LPCTSTR pSrcResource,
  __out LPDIRECT3DTEXTURE9 *ppTexture);

Another possible approach is to utilize single function and interpret given pointer as either string with file path or a direct memory buffer, depending on some flags. That's the way you can load sound samples in FMOD library:

FMOD_RESULT System::createSound(
  const char * name_or_data,
  FMOD_MODE mode,
  FMOD::Sound ** sound

Where name_or_data is "Name of the file or URL to open, or a pointer to a preloaded sound memory block if FMOD_OPENMEMORY/FMOD_OPENMEMORY_POINT is used."

3. That's more flexible, but sometimes an object is so big that it's not efficient or even possible to load/uncompress/download its full contents into memory before creating a real resource or do some processing. What's needed is an interface to process smaller chunks of data at time. One of ways to do it is defining an interface with callbacks that the library will call to query for additional piece of data. Then we can implement this interface to read data from any source we wish, whether simple disk file, compressed archive or a network socket. When we want to load an object, we call appropriate function passing pointer to our implementation of the interface. During this call our code is called back and asked to read data. For example, that's the way we can load sounds in Audiere library. The interface for reading data is:

class File : public RefCounted {
  ADR_METHOD(int) read(void* buffer, int size) = 0;
  ADR_METHOD(bool) seek(int position, SeekMode mode) = 0;
  ADR_METHOD(int) tell() = 0;

4. A step futher towards more flexibility and generality is the concept of streams, like from Java, C# or Delphi. These object-oriented languages define in their standard libraries an abstract base class for input (for reading data) and output stream (for writing data) that can be implemented in many possible ways. For example, Java's InputStream class defines methods:

void close()
int read()
int read(byte[] b)
int read(byte[] b, int off, int len)
void reset()
long skip(long n)

Many derived classes are provided. Some of them read/write data from real sources like file or network connection, while others process data (for example compress, encrypt) and pass them to another stream. This way a chain of responsibilities can be created where we write data to a stream that compresses them and pass them to another stream, which encrypts them and passes them to the one that does buffering and finally write the data to the stream that saves it to a file. It's my favourite approach right now, although it has a drawback - an overhead for virtual method calls in each stream for each piece of data read or written. This inefficiency can be minimized by controlling granularity - processing a buffer of reasonable size at time, never byte-after-byte.

5. Finally, there is the most direct, low-level approach which is also most flexible and efficient, but at the same time very difficult to use properly. I'm talking about a single function that takes pointers to input and output buffers, as well as some structure containing current state and processes a piece of data. It consumes some/all data from input buffer (by advancing some pointer or counter) and produces new data to the output buffer. There are no callbacks. The interface is neither "push" (where we write data) or "pull" (where we read data), but both at time. That's the way zlib compression library works (which I complained about here, in Polish), as well as LZMA SDK (which I described here).

typedef struct z_stream_s {
  Bytef  *next_in;  /* next input byte */
  uInt   avail_in;  /* number of bytes available at next_in */
  uLong  total_in;  /* total nb of input bytes read so far */

  Bytef  *next_out; /* next output byte should be put there */
  uInt   avail_out; /* remaining free space at next_out */
  uLong  total_out; /* total nb of bytes output so far */

  char   *msg;    /* last error message, NULL if no error */
  struct internal_state FAR *state; /* not visible by applications */
} z_stream;

ZEXTERN int ZEXPORT deflate OF((z_streamp strm, int flush));

Comments | #software engineering #c++ Share

# A Random Thought on Over-Generalizing

Jul 2010

I don't post next part of the description of my reflection system yet. Instead I just want to share a small thought that came to my mind today. It's about over-generalizing, over-engineering, writing overly-abstract code or however you call it. Using too much OOP is not only the matter of code perfomance, but something deeper, more ideological. Interesting blog entries about this topic are: 5 Stages of Programmer Incompetence (see "The Abstraction Freak" paragraph) and Criminal Overengineering @ yield thought, Smartness overload and Smartness overload - addendum @ mischief.mayhem.soap. A Counterpoint can be found at cbloom rants. And finally here is my idea:

It is a vicious circle. Here is how it works:

But I believe this is true only to some degree. We obviously need a general, universal code sometimes not to do the same, monotonous or error-prone work over and over again. That's why we create and use libraries. And that's why I've coded my reflection system :)

New: I can see similar vicious cycle in a programming language development, inspired by an article I've read today: Google engineer calls Java and C++ too complicated. For me it looks like this: software engineers, companies and committees develop very sophisticated programming languages because they want them to be as much general and universal as possible so developers don't have to learn and use many specialized languages for different purposes. Developers don't like to learn many new programming languages because they have bad experience from learning and using such universal, spohisticated languages.

Comments | #software engineering #philosophy Share

# Programmer Incompetence?

Jun 2010

Today I'll be a little more philosophical, of course in terms of programming philosophy ;) I manage a long list of Web addresses I'd like to visit in my free time (which I gather mostly from Twitter) and today I've visited these two places:

They reminded me of my old thought that on the way of mastering the art of programming there is no such thing as diversity - no place for personal preferences that would be only a matter of taste, like favorite music. There is usually only one correct path. All the good practices and solutions are just a function of what do you want to do and what technologies do you use. I constantly learn stuff, interact with programmers better than me as well as observe and talk to these who just start their adventure in the programming world. From all these experiences I can see that there usually is a single, correct solution or the way of thinking about simply anything. The one who doesn't agree with it is just not "mature" enough to be able to see that. Another sad truth is that this "maturing" process cannot be accelerated - one can read about the "correct" way but just rejects it until he will come to it by himself.

I especially like the 5 Stages of Programmer Incompetence entry. Here is my self-diagnosis against its points:

Comments | #software engineering #philosophy Share

# Some Thoughts about Library Design

May 2010

Much has been said about designing good user interface, whether for desktop applications, websites or games. There are whole books available about GUI design and even this year's IGK-7'2010 conference featured two lectures about the interface in games. But what about interfaces for programmers?

I can't find much about the rules of good library API design and I believe there is much to say in this subject. I only know The Little Manual of API Design written by Jasmin Blanchette from Trolltech/Nokia, one of the creators of Qt library (thanks for the link Przemek!). There is also a blog entry about Math Library, which is quite interesting. Inspired by it, I've came up with a general thought that you cannot have all the following features when designing a library and its API, you have to choose 2 or 3 of them and make some compromise:

I think some patterns and best practices as well as some anti-patterns could be found when you look at interfaces of many libraries. Maybe I'll post some more of my thoughts on this subject in the future. In the meantime, do you know any other resources about API design?

Comments | #philosophy #software engineering #libraries Share

# The Concept of Immutability and Descriptor

Nov 2009

An object of some class represents a piece of data, chunk of memory or other resource along with methods to operate on it. It should also automatically free these resources in destructor. But how should modifying these data look like? There are two possible approaches. As an example, let's consider a fictional class to encapsulate Direct3D 9 Vertex Declaration (I'll show my real one in some future blog entry). A Vertex Declaration is an array of D3DVERTEXELEMENT9 structures, which can be used to create IDirect3DVertexDeclaration9 object. First solution is to define class interface in a way that data inside can be modified at any time.

class MyMutableVertexDecl
  // Creates an empty declaration.
  // Frees all allocated resources.
  // Copies data from another object
  void CopyFrom(const MyMutableVertexDecl &src);
  // Deletes all internal data so object becomes empty again.
  void Clear();
  bool IsEmpty() const;
  // I/O (Serialization)
  void SaveToStream(IStream &s) const;
  void LoadFromStream(IStream &s);
  // Reading of underlying data
  size_t GetElemCount() const;
  const D3DVERTEXELEMENT9 & GetElem(size_t index) const;
  // Modification of underlying array
  void SetElem(const D3DVERTEXELEMENT9 &elem, size_t index);
  void AddElem(const D3DVERTEXELEMENT9 &elem);
  void InsertElem(const D3DVERTEXELEMENT9 &elem, size_t index);
  void RemoveElem(size_t index);
  IDirect3DVertexDeclaration9 * GetD3dDecl() const;
  std::vector<D3DVERTEXELEMENT9> m_Elems;
  IDirect3DVertexDeclaration9 *m_D3dDecl;

This approach seems very nice as you can create your object any time you wish and fill it with data later, as well as change this data whenever you need to. But at the same time, a question emerges: when to (re)create "destination" IDirect3DVertexDeclaration9 from the "source" D3DVERTEXELEMENT9 array? Each time the array is modified? Or maybe each time the IDirect3DVertexDeclaration9 is retrieved? Optimal solution for the interface above would be to do lazy evaluation, that is to recreate IDirect3DVertexDeclaration9 whenever it is retrieved for the first time since last time the D3DVERTEXELEMENT9 array have been modified. But...

Read full entry > | Comments | #software engineering #directx #.net #physics #c++ Share

# About the Guy Who Made Love

Aug 2009

Today I want to talk a bit about what's the dream of almost every passionate game developer. It seems very hard or almost impossible to achieve, but younger amateurs still hope that they will manage to do it someday. Of course I'm talking about making a 3D MMO game.

As it turned out for me today (thanks for the link KriS!), it actually IS possible. I'm talking about the game called Love written entirely by one person - Eskil Steenberg. He have coded all the software from modeling tools through network protocol and renderer until game mechanics. To see it working I recommend watching these videos. The game is powered by his engine called Quel Solaar, which is actually available for download.

I must admit I haven't been impressed so much for a long time. I suppose the amount of time and passion that had to be put into this code is enormous. Graphical style and gameplay, as well as the user interface of his tools are very unusual and surprising. And all of this is made by one guy...

I recommend watching his lecture from this year's Assembly party titled Developing the technology behind "Love". You can see many technical details and if you don't want to watch the entire one hour video, at least watch the beginning (where he talks about his "smarter way of doing things") and the ending (where he expresses his thoughts about the value of good tools).

BTW it's also nice to watch new videos from GC 2009 of the CryEngine 3. "What you see is what you play" and instant asset update (including textures) - that's how good game editor should look like :)

Comments | #software engineering #networking #web #rendering #games #events #engine #demoscene #philosophy #tools Share

# How Do People Write Bad Code

Aug 2009

Recently yarpen posted some thoughts about source code scalability on his blog. His blog entries are always very valuable and this time he touched broader subject which is connected to software antipatterns.

I want to expand this subject further and think a bit about how is such bad code created (we all agree that non-scalable code is bad, right? :) In my personal opinion it's generally caused by the lack of thinking. For example, if a coder have to code something (or, how some people would like to say: to solve a problem), he should first consider what possible solutions are. This may be choice of a technology, programming languages and libraries, some design patterns, algorithms, data structures or even names for some objects. If he doesn't think about it, he just selects the first option that came to his mind without considering alternatives and thus risks that he didn't make the best possible decision.

Next, the coder should design his solution. Designing is all about thinking before doing. If he skips this step, he may meet during his coding a situation when sees he came the wrong way and now he must refactor or even delete big part of his code and then just... rethink it :) I like the saying It's better to think twice before you start coding than to code twice before you start thinking. :)

The same applies to all changes made to an existing code. If somebody have to change something in a program, he'd better review the code carefully to discover and understand its architecture. If he just starts making changes to the code, he risks that he will break some hidden assumptions in some aspects of the code (like variable values, objects lifetime, order of function calls etc.) and thus create bugs.

The reason behind writing poor code may be rush. If a coder wants to code something as quick and simply as possible (or his boss tells him to do it this way), he will do ugly hacks and never refactor anything, even if some refactoring is necessary at this point. Of course this it to bad code and will backfire in the future, when the code will have to be changed and it will turn out to be totally unscalable. So we obviously see that such attitude is just the lack of thinking.

But this problem may also be something deeper. I can see that some coders just always write bad, unscalable code by their nature. I don't want to offend anyone, but I think maybe that's the crucial difference between junior and more experienced programmers. To work as a programmer, everyone needs to "speak fluently" his programming language to freely express his thoughts using it, but experienced programmers are also able to see a bigger picture - some invisible assumptions and more abstract concepts behind the code. They see words like "handle", "manager", "alignment", "serialization", "callback" and hundreds of other terms (anyone thought about creating coder's dictionary with words like these?) and just knows what hides behind them.

Three days after yarpen's entry there appeared a blog entry about code documentation on EntBlog. It describes two main types of documents - API documentation and technical articles. I fully agree with this text. I value documentation very much and I think that good documentation is another piece of this big puzzle.

They say that there are two types of people: these who make backups and these who will make it :) I think it's the same with writing good code and documentation. It's just the matter of thinking forward.

Comments | #software engineering #philosophy Share

Pages: > 1 2 3 4 5 >

[Stat] [STAT NO AD] [Download] [Dropbox] [pub] [Mirror] [Privacy policy]
Copyright © 2004-2019