2015/09/29

Why WaitForMultipleObjects() is bad and why useful things seems to get lost in dark corners of internet much easier than pictures of funny cats

Some time ago, I made a small research of embedded RTOS. I also published my personal opinion about them in my second blog.

Recently I'm preparing a newspaper article and just wanted to recall some interesting point of view why Microsoft Windows WaitForMultipleObjects() functions family is a bad idea. And guess what ... I spend almost hour to find this post in the internet ... ughh

So since things like to get lost in dark corners o internet ... I insolently decided to copy-paste this post :D ... the original post can be (as time of writing this note) found here.


Author of following post is  by Kaz Kylhe Mon, 19 Apr 1999 04:00:00
Original thread name: "pthreads and waiting for MULTIPLE conditions"
Site: http://www.verycomputer.com/5_c6341d98f68da45e_1.htm


Andy Levin was asking for:

I am porting some NT work I wrote a while back to the UNIX world and have the need to use pthreads.
Is there any functionality in the pthread libraries (or elsewhere) that allows you to wait for MULTIPLE conditions to occur.

Kaz Kylhe replays:

This is not needed. Think about it. You have one thread and you want to
put it to sleep. In a typical OS kernel, all processes or threads sleep
in one ``place'' at a time, such as a supend queue and whatnot.

What you are asking for is like wanting to sleep in two or more beds
at the same time. This, of course, is only possible if you are the president of
the United States.

> I am looking for functionality similar to the Win32 function WaitForMultipleObjects()

That Win32 function is seriously misconceived and should be avoided even in
Win32 programming. It has problems. For example, in the case that any *one* of
the objects can wake up the thread, there is potential starvation by one object
that is constantly going off, since the function only reports one object during
one call. (Contrast this to the much cleverer POSIX select() function that
gives you a bitmap representing objects that are ready, thus avoiding
starvation.) Another problem is that this function does not scale beyond 64
objects. If you need to wait for more events or what have you, you have to
launch additional threads. For example, if you wanted to wait for 256 objects
in Win32, you would launch four threads. Each thread would wait for 64
objects. And your ``parent'' thread would do a WaitForMultipleObjects on the
four thread handles. Ugly, ugly, ugly.

Programs that use WaitForMultipleObjects often break encapsulation; you often
see a technique whereby software modules export events, in effect saying ``here
is my event, I will set it when something interesting happens''. Needless
to say, this is hard to port.

A much better approach is to keep events buried in the implementation of
something; notification can be provided much more efficiently by direct message
passing. One object should not know about the internal events and threads of
another object.

For example, suppose that you have a thread that needs to wait until some other
object completes some task, or a timer goes off. One way to do this under Win32
might be to have two events. Your thread waits on both of them using
WaitForMultipleObjects. The events are exported to other objects (such as the
timer or the worker) which do a SetEvent. In other words, the use of Win32
events is made explicit in the interfaces between objects, a clearly
portability mistake.

A better way to do this might be to have registered callbacks. When the timer
goes off, a callback is invoked back to the waiting object. When the worker
object is done doing something, a different callback is invoked. Both
callbacks can signal the same Win32 event, so your thread just needs to wait on
that single one. This model is readily supported by POSIX condition variables,
and is easier to port among different threading platforms, because it is based
on a message passing abstraction that is directly supported by C and C++,
namely function calls.


>allows you to specify an array of synchronization objects and conditions wait for all, wait for any, etc) to wait for.

There is no etc: you can wait for all, or you can wait for any (with
potential starvation, requiring you to constantly permute the array
passed to WaitForMultipleObjects).

2014/01/03

CPU architecture

http://ootbcomp.com/docs/belt/

2012/01/24

History

Today with my several colleagues  we recall old times of Commodore 64 and such stuff. During the discussion I mentioned that I had a mentor on my university and he sometimes told me such stories (which were obviously quite older), like about times of punched cards and Odra computers.

It is funny that some of those stories continue in present times in form of joke or connotations:

POKE and POKE cheat or Killer POKE
Halt and catch fire
or even such common a BUG (first bug is even stored in museum ;))



2012/01/02

Microsoft CEO


Microsoft CEO - No comments :D

2011/12/27

Usefull C macros

By couple of years of professional programming, I have collected some useful C macros which I use on daily basis. I don’t imagine programming without them. I figure out that not many people use such macros, or even if they use similar ones, their version is vulnerable for macro parameter side effects and signed overflow hacks.

Because my macros are much better ;) I decided to share. Some of them are my pure creation, some of them are copied for other places (like Linux kernel). Because of that I am obliged to release them under GPL (I hope that nobody will sue me for 4 line of C code even if he/she came onto the same obvious idea).

Some of those macros are just renaming for more robust code when you plan to migrate between different compilers. Some other are useful during C kind object-like programming. At least  some of them save some repeat coding ;)

/**

* Macro mask the "unused parameter" warning when using -Wall -Wextra compiler
* options. Those compiler options should be used to prevent typical C coding
* mistakes while unused macro allows to silent the waring when we really what
* to not use all of function parameters
*/#if defined(__GNUC__)
# define unused(x) UNUSED_ ## x __attribute__((unused))
#elif defined(__LCLINT__)
# define unused(x) /*@unused@*/ x
#else
# define unused(x) x
#endif

/** likely/unlikely macro is to provide the compiler with branch prediction
* information. By explicitly giving such info you may instruct compiler
* to produce code optimized for more probable use case. As result compiler
* will move the code sections for full utilization of CPU instruction cache
* and jump elimination (CPU pipeline flush).
*/
#if __GNUC__ >= 3
#define likely(x) __builtin_expect(!!(x), 1)
#define unlikely(x) __builtin_expect(!!(x), 0)
#else
#warning Compiler is not supporting all extensions, some optimization will have no effect
#define likely(x) (x)
#define unlikely(x) (x)
#endif

/** always_inline tells GCC to inline the specified function regardless of whether optimization is enabled.
* deprecated tells you when a function has been depreciated and should no longer be used. If you attempt to
use a deprecated function, you receive a warning. You can also apply this attribute to types and

variables to encourage developers to wean themselves from those kernel assets.
* __used__ tells the compiler that this function is used regardless of whether GCC finds instances of calls

to the function. This can be useful in cases where C functions are called from assembly.
* __const__ tells the compiler that a particular function has no state (that is, it uses the arguments

passed in to generate a result to return).
* warn_unused_result forces the compiler to check that all callers check the result of the function. This

ensures that callers are properly validating the function result so that they can handle the appropriate

errors. */
#if __GNUC__ >= 3
#define __always_inline__ __attribute__((always_inline))
#define __deprecated __attribute__((deprecated))
#define __attribute_used__ __attribute__((__used__))
#define __attribute_const__ __attribute__((__const__))
#define __must_check __attribute__((warn_unused_result))
#else
#define __inline__
#define __deprecated
#define __attribute_used__
#define __attribute_const__
#define __must_check
#endif

/**
* Prefetch portable macros for cache preload by buildin uC instructions.
* This macro allows to produce more optimized code on CPU equipped with cache.
* Placing prefetch instruction before code section that use some data may
* influence on execution time, since procesor will load those data to the
* cache before the instructions request them. Without prefetch, data
* load will be done when instructions require mentioned data, while CPU will
* be staled until those data will be loaded to cache.
*/
#if __GNUC__ >= 3
#define prefetch(x) __builtin_prefetch(x)
#else
#define prefetch(x)
#endif

/**
* Restrict keyword can be used to variables which are pointers and which points
* to content not accessible by other pointers or parameters.
* The main purpose of this keyword is to instruct the compiler that content
* under the particular pointer will not be changed by other way than explicit
* pointer reference. This allows the compiler to produce more optimized code.
*
* Without restrict keyword, compiler have to assume that any two pointers may
* point to the same memory location. Because of that compiler cannot reuse
* temporary values since it may be invalidated by any other indirect (pointer)
* write access.
*/
#if __GNUC__ >= 3
#define restrict __restrict__
#else
#define restrict
#endif

/** \note gcc-4.0.1/gcc/Return-Address.html On some machines it may be
* impossible to determine he return address of any function other than the
* current one; in such cases, or when the top of the stack has been reached,
* this function will return 0 or a random value. In addition,
* __builtin_frame_address may be used to determine if the top of the stack
* has been reached, for compatibility our macro calleraddr returns only the
* return addres of current function */
#if __GNUC__ >= 3
#define calleraddr() __builtin_return_address(0)
#else
#define calleraddr() NULL
#endif

/**
* Common macro that allows to calculate the offset of field in structure
*
* @param _type Type of parent
* @param _member Name of member inside parent
*
* @return Offset in bytes (size_t) of member from the beginning of parent.
*/
#ifdef __compiler_offsetof
#define offsetof(_type,_member) __compiler_offsetof(_type, _member)
#else
#define offsetof(_type, _member) ((size_t) &(((_type *)NULL)->_member))
#endif

/** Common macro that allows to get size of member in structure or union */
#define sizeoffield(_type, _member) (sizeof(((_type *)NULL)->_member))

/**
* Common macro that allows to calculate pointer to parent,
* from pointer to member, name of the member inside parent and parent object type
* Using of temporary pointer _mptr is necessary to prevent macro side effects for
* operands like pointer++
*
* @param _prt Pointer to member
* @param _type Parent object type
* @param _member Name of the member inside parent object
*
* @return Pointer to parent
*/
#define container_of(_ptr, _type, _member) ({ \
const typeof( ((_type *)0)->_member ) *_mptr = (_ptr); \
(_type *)( (char *)_mptr - offsetof(_type,_member) );})

/*
* Common min macro with strict type-checking,
* returns the smaller value from two operands
*
* Strict type checking in important aspect of secure code,
* (sign type mixed checking is common source of exploitable bugs).
* Using of temporary values is necessary to prevent macro side effects for
* operands like variable++
*
* @param _x First value
* @param _y Second value
*
* @return Smaller of two passed values
*/
#define min(_x, _y) ({ \
typeof(_x) _min1 = (_x); \
typeof(_y) _min2 = (_y); \
(void) (&_min1 == &_min2); \
_min1 < _min2 ? _min1 : _min2; })

/*
* Common max macro with strict type-checking,
* returns the greater value from two operands
*
* Strict type checking in important aspect of secure code,
* (sign type mixed checking is common source of exploitable bugs).
* Using of temporary values is necessary to prevent macro side effects for
* operands like variable++
*
* @param _x First value
* @param _y Second value
*
* @return Greater of two passed values
*/
#define max(_x, _y) ({ \
typeof(_x) _max1 = (_x); \
typeof(_y) _max2 = (_y); \
(void) (&_max1 == &_max2); \
_max1 > _max2 ? _max1 : _max2; })

/**
* Macro used to calculate the ceiling(x/y), macro is type sensitive.
* Macro cannot be used with floating point types,
* for those please use ceil function from math.h
*
* @param _x dividend
* @param _y divisor
*
* @return ceil(_x/_y) with the same type as _x operand
*/
#define ceil_div(_x, _y) ({ \
typeof(_x) __x = (_x); \
typeof(_y) __y = (_y); \
(void) (&__x == &__y); \
typeof(_x) _rem = __x % __y; \
typeof(_x) _div = __x / __y; \
(_rem > 0) ? (_div + 1) : _div; })

/**
* Version of ceil_div without strict type checking, use with care
* Warning this version has common macro side effects for operands which use
* ++ or --
*/
#define ceil_div_nocheck(x, y) (((x) / (y)) + (((x) % (y)) ? 1 : 0))

/**
* Version of min without strict type checking, use with care
* Warning this version has common macro side effects for operands which use
* ++ or --
*/
#define min_nocheck(x, y) (((x) < (y)) ? (x) : (y))

/**
* Version of cmax without strict type checking, use with care
* Warning this version has common macro side effects for operands which use
* ++ or --
*/
#define max_nocheck(x, y) (((x) > (y)) ? (x) : (y))

/**
* Macro clamps the value to the given range, macro performs strict type checking.
*
* Strict type checking in important aspect of secure code,
* (sign type mixed checking is common source of exploitable bugs).
* Using of temporary values is necessary to prevent macro side effects for
* operands like variable++
*
* @param _val Clamped value
* @param _min Lower bound
* @param _max Upper boud
* @return Clamped value
*/
#define clamp(_val, _min, _max) ({ \
typeof(_val) __val = (_val); \
typeof(_min) __min = (_min); \
typeof(_max) __max = (_max); \
(void) (&__val == &__min); \
(void) (&__val == &__max); \
__val = __val < __min ? __min: __val; \
__val > __max ? __max: __val; })

/**
* Macro clamps the value to the given range using val param type
*
* This macro does no type checking and uses temporary variables of whatever
* type the input argument 'val' is. This is useful when val is an unsigned
* type and min and max are literals that will otherwise be assigned a signed
* integer type.
*
* @param _val Clamped value
* @param _min Lower bound
* @param _max Upper boud
* @return Clamped value
*/
#define clamp_val(_val, _min, _max) ({ \
typeof(_val) __val = (_val); \
typeof(_val) __min = (_min); \
typeof(_val) __max = (_max); \
__val = __val < __min ? __min: __val; \
__val > __max ? __max: __val; })

/**
* Macro returns number of table elements
*
* @param _table Table pinter
* @return Number of table elements, the return type is size_t
*/
#define element_cnt(_table) (sizeof((_table)) / sizeof((_table)[0]))

/**
* Macro rounds up the address to platform basic type alligment rules.
* As an example on 32 bit platforms it will return address aligned up to 32 bits.
*
* @param _addr Unaligned address
* @return Aligned address
*/
#define addr_allign(_addr) \
( (void*)(divtop_nocheck((unsigned long)(_addr), sizeof(long)) * sizeof(long)) )

/**
* Macro rounds up the size to platform basic type alligment rules.
* As an example on 32 bit platforms it will return size aligned up to 32 bits.
*
* @param _addr Unaligned size
* @return Aligned size
*/
#define size_allign(_size) \
( (size_t)(divtop_nocheck((size_t)(_size), sizeof(long)) * sizeof(long)) )

/**
* \brief Malloc with out of memory protection and memory cleaning
*/
#define safe_alloc(_type) ({ \
_type *ptr = malloc(sizeof(_type)); \
assert(NULL != ptr); \
memset(ptr, 0, sizeof(_type)); \
ptr; })

/**
* \brief free with "reference after free" protection
*/
#define safe_free(_ptrtype) \
do { \
memset(_ptrtype, 0xAB, sizeof(*(_ptrtype))); /* 0xAB is a safe patern */ \
free(_ptrtype); \
(_ptrtype) = NULL;\
} while(0)

/**
* Compile-time assertion useful for check done during the compilation time.
* It provides similar functionality to assert macro (which is calculated
* during the execution time). static_assert is evaluated during the compilation
* stage (not during the preprocessing). Because of that it may be used with
* conjunction to cost variables or C expresions like sizeof(type).
*
* \remarks It relies on 'unused' macro and assumes that 'unused' macro
* adds "UNUSED_" prefix to name. It might generate "unused variable"
* warning in case of disabled 'unused' macro.
*/
#define static_assert(expr) \
static const char unused(unique_name [(expr)?1:-1]) = {'!'}
#define unique_name make_name(__LINE__)
#define make_name(line) make_name2(line)
#define make_name2(line) constraint_ ## line

2011/12/20

Modular embedded development with mbeddr C previewed

Check this mbeddr C Extension Guide. It seem to be a try to deal with embedded development problems which came from C lang.

2011/12/15

Implicit type casting in C - aka signed overflow bug

I just made small discovery, that any ANSI C compiler by default don't warn about implicit conversions. This convention accustomed many programers to freely use of signed and unsigned variables. The most common practice is to pass the signed variable as a parameter of function which expect the unsigned one. Another bad habit is to compare the signed and unsigned variables. This is commonly known source of bug (at least in hacker world) ;)

For instance:
$ cat main.c
#include
#include

void func(size_t param)
{
    printf("done %zu\n", param);
}

int main()
{
    int param = -1;
    func(param);
    return 0;
}

$ gcc -o main -Wall -Wextra main.c

$ gcc -o main -Wall -Wextra -Wconversion main.c
main.c: In function 'main':
main.c:12: warning: passing argument 1 of 'func' with different width due to prototype

As we see even with -Wall -Wextra compilation parameters, there will be no compilation warning. The warning will appear only if we explicitly request -Wconversion option.

Signed param  will be passed to func with implict cast to unsigned type. This in certain circumstances may cause serious security flaw. In code above we will get 18446744073709551615 (on X64) as a result instead of -1. (not very insecure but I'm sure that you get the base).

More destructive effect of this rule can be presented by following code:

void func(size_t param)
{
   if(param < 5)
      printf("param less then 5\n");
   else
            printf("param grater then 5\n");
}

In we run this code we will see "param greater then 5", which is not what we expect, since we pass -1 as function param. This can happen if we don't make enough attention for the variable types before passing it to function. In this particular case if we want to remain the original param value to be signed, we have to make additional value checking code (assert) before we call the func.

It is worth to mention that with compiler flags -Wall -Wextra we will warn about signed/unsigned compare operation, while it will not warn about implicit type casting for function params. Like in below code, we will get warning for line with "if" while we will not get a warning for line with func(-1, 5).

void func(size_t param, size_t limit)
{ 
   if( param > limit ) 
      printf("greater\n");
}

int main()
{
    int param = -1;
    unsigned int limit = 5;

    if( param > limit )
        printf("greater\n");

    func(-1, 5);

    return 0;
}