Edgar Huckert
 

Programming in D

Contents:

Some weeks ago I have decided to start all new projects rather with the programming language D than with C++. My reasons are the following:

Note that I am not completely satisfied with D: it has some features that have their motivation in academic discussions and is thus also more complex than needed. Unfortunately the current version V2.x is largely incompatible with D V1.x so that I had to rewrite or modify all my old D programs.

Be careful with the official D language reference and documentation: it is often academic and confusing.

Meanwhile (2015) a good tutorial has become available: Cehreli, Ali: "Programming in D. Tutorial and reference", San Francisco (2015). It concentrates on the language D. Unfortunately the D standard library Phobos is not in the focus of this book. I hope that my sample programs help to fill some gaps in the book by Cehreli.

I make many comparisons in the following text and in the program comments with C or C++. You may find parallel constructs in other programming languages like Java or PHP etc.

All my sample programs have been tested under Windows and under Linux Ubuntu using the D compiler (program dmd) on the command line. The Phobos library is used exclusively. The demo programs are not meant to be models for academic discussions.


Loops

In my sample program loops.d I demonstrate 7 types of loops. D allows goto - a statement that I use here for one loop type. Most of the loop types are identical with loops in C or C++.

The foreach loop type is however idiosyncratic in D: the foreach loops replaces in many situations the iterators in C++ (which does not mean that you can't write your own iterators in D). I show here a foreach loop over an array. A similar foreach loop over a DList of tuples (a DList is a container type) can be found in my sample program lists.d.


Strings and number conversions

Strings in D are a special type of array, i.e. they are class instances (not simple arrays like in Standard C). Important: they are immutable arrays of char elements, i.e. you cannot assign a value to an individual element. Non-destructive (read) access to individual elements is allowed. Complete assignments are also allowed. If you plan to change individual elements then you have to convert the string like in this example:
 
string s1 = "abcd";
// s1[2] = 'x'; not allowed
char ch = s1[3]; // read for indiv. element OK
char [] s2 = s1.dup; // OK
s2[2} = 'x';
ubyte [] ub2 = cast(ubyte[])s1.dup; // using ubyte instead of char OK
s1 = cast(string)s2; // make a string again

D has the same problem as C++: it tries to preserve compatibility with C. You can invoke C library functions and hence the existence of (zero terminated) C strings is required. This - the immutability and the new problems with multibyte (Unicode) characters - makes the chapter "strings in D" more complex than expected. I don't test multibyte characters (classes dchar, wchar, dstring, wstring) here. Even with single byte characters (DOS codepage 850 encoding) I had problems when including German umlauts in my sample program. Please note that I found at least one incompatibility between Windows and Linux (see the comments) in my sample program.. See my dictionary program to see how charset problems (codepage 850 to UTF-8) can be handled in IO from console and file.

The basic string features tested in my sample program for D strings:

Note that there are some traps if you use the C like conversions à la sformat(): see my comments in the sample program.


Input, output, directories, Unicode

As in C and C++ IO is not part of the D languages but rather part of the libraries. Input and output are handled in the Phobos std.stdio and std.file import files. The more traditional, C like functions are defined in std.stdio. The functions include traditional functions like read(), write(), seek(), tell (tell is a property), open() and close(). The only real new function is rawRead() which must be used for raw (=binary) files. The Phobos libray does not contain any IO style corresponding to C++ streams (cin, cout, cerr etc.) - may be some other library has such concepts. I am not sad about this missing IO style.

The other import file - std.file - can be used to read and write memory portions in a single pass, i.e. without using chunks of data. This IO style can handle binary files. It offers however no fine granular access into files as it has no methods like seek() or tell().

Note that the different IO styles produce different types of exceptions.

My test program illustrates both types of IO with UTF-8 files and binary files. It also shows simple IO on the console, i.e. via files stdin and stdout.

The handling of directories is fortunately very simple in D. My test program illustrates the iteration through all files of a directory including all files in all subdirectories. It outputs two of the essential attributes for each object found during the traversal: the distinction between (normal) files and directories and the modification date. If you compare this simple program with the usual directory traversals in C you will be glad to see that no recursion is necessary.

If you read frequently simple text files in D you may encounter Unicode or UTF8 exceptions. I ran in this situation as I have very old text files from the MSDOS or early Windows days. The same problem may appear if you read input from the Windows7 console. The reason for these exceptions is simple: the strings in D and its libraries can be used for Unicode-based character sets. Unicode (1, 2, 4 or multi-byte encodings) should not be confused with ISO-8859-x encodings (ex. ISO Latin 1, 1 byte encoding). I include here a simple dictionary program that catches these exceptions. Unfortunately the correct handling and conversion of character sets is -also in D - rather complicated and sometimes confusing. See my article on codepage conversion.

Dictionaries for natural language processing require multimap container classes as the same key may appear with multiple contents (English "work" may be a noun or a verb).. I use here simple associative arrays that cannot handle directly multiple identical keys. My solution is simple: I concatenate the contents after adding a separator so that the returned category may be in fact a sequence of categories that must be parsed a little bit.


Date, time, elapsed time

Getting the current time, the actual date and the elapsed time is rather simple in Windows. It is more complex in the Unix C library. In D it is even more complex as the documentation is not very clear. Here is my sample program that demonstrates how date and time can be obtained. It also shows basic manipulation (the roll() call). Two methods are demonstrated showing how the execution time for an external process can be measured.


Structs, classes, class derivation, polymorphism, Traits

Structs are - like in C++ - a simplified edition of classes. Structs are value types - classes are reference types! My test program illustrates the definition of structs and access to the member data and member functions. It also demonstrates the use of constructors for structs. Note that the use of typedef (as in C and C++) is not allowed.

Classes are also very close to the classes in C++. The main difference between D and C++: the explicit pointer notation (using '*' in C++) is not required. I demonstrate here a simple case of class derivation including the invocation of the super class and the call of member methods and the (implicit) call of a super class method. The invocation (explicit using delete and implicit when the class variable goes out of scope) of destructors is shown. The program contains also a simple copy constructor as used in C++.

A special case of polymorphism that I use heavenly is shown: the convenient member function is called via a pointer to the base class. This base class pointer is found in an array (could also be a container like DList)

Finally two samples for Traits (sizeofClassInstance and allMembers) are tested. Traits are similar to RTTI in C++ or reflection in Java. Whereas RTTI and Java reflection offer run time information, Traits in D offer only compile time information. I use here simple Traits using the keyword __traits. There is also a more elaborate D module called std.traits.


Working with lists and containers

I concentrate on doubly linked lists - a type of container that I use often in my projects. Lists are useful when you need functions returning more than one value (from a mathematical point of view this is forbidden - but they can return a single list). Lists are also very useful to collect the output of database searches.

Unfortunately the container type called DList (see package std.container) that I tested is not the most versatile container type in D (see my comments in the source code). The documentation is bad - it doesn't mention neither lacking features like the direct access via index ("[i]") nor lacking properties like ".length". My sample program demonstrates very simple lists, lists containing tuples and lists containing class instances. D has no elaborated iterator concept like C++. It uses generalized foreach loops instead. In my sample code you will find a foreach loop that retrieves all elements of the DList.

In Phobos you will find in the package std.container other container types like SList (singly linked list) or RedBlackTree - this latter class allows also duplicate keys and can work with a user defined hash function. If you come from C++/STL: here is no name equivalent for "map" or "multimap". Some users on the Internet recommend to use dynamic associative arrays instead of maps - I must agree. See my dictionary program for the use of dynamic associative arrays.


Function pointers, function templates, delegates

I assume that the normal use of functions is known: there is nothing special compared with C or C++.

Function pointers are also - except the syntax - very similar to C or C++. One warning however: don't confuse function pointers with delegates! Function pointers are a much simpler concept than delegates. As shown in this sample program, a function pointer is a variable that can be assigned an address of a function (routine, procedure, method). This variable can be used to call the actually assigned routine in the normal call syntax. The "address-of" operator & must be used for the address of the target routine. You have to use the keyword function for the declaration as in this sample:

      int function(int) pFunc; // declaration
      pFunc = &fooGlobal; // get address of function
      result = pFunc(6); // call

Function pointers are useful when you have to decide at run time which function should be called. Function pointers are - unlike delegates - basic types. They have no useful properties in D. Compared to C and C++ the syntax is much easier and more intuitive.

The difference between function pointers and delegates on a 32 bit machine:

As in C and C++ a function pointer is a simple address pointing to a function or routine. delegates can be used too as pointers to functions. A delegate however is more complex: it has Delegates work normally in the context of classes (object oriented programming). Normally you can use delegates pointing to internal functions and to member functions. There is however a trick (see my sample program) to get a delegate for a global routine not associated to a class instance: use std.functional.toDelegate(). Another advantage of delegates is their closure-like behaviour. We do no discuss that here.

Unfortunately the use of delegates is not very simple and has many restrictions. See this sample file where I have tested function pointers and delegates in different contexts.

Function templates are useful if you want to write the same function body for varying arguments. In my sample a small routine can be invoked with integer, string and double arguments. The syntax for the declaration is very similar to class templates. For more complex applications you should consider virtual methods in classes that are related via derivation.


Sockets

UDP

The first socket test program deals with UDP (datagram) sockets. It sends a simple message to an UDP listener (see program udpreceiver.d). You can specify the message, the host and the port on the command line. Example (must be run in two console windows):

    udpreceiver 9514
    udpsender "Hello E.Huckert" localhost 9514

I use UDP heavily for logging (see here). These two D programs can serve as the base for an UDP logger. When testing be sure that you use the same UDP port in both programs and that this port is not blocked by your firewall.

There is nothing special with UDP sockets in Phobos except the connect() call: everybody knows that UDP doesn't build a connection. This call is therefore a little bit strange - but seems to be necessary. A second important point: in contrast to the official Phobos documentation the UDP socket (receiver side) is not blocking!

TCP

I have prepared two very simple TCP programs showing the essential classical techniques: a TCP sender for UTF8 messages and a TCP receiver.. Both programs use the blocking calls as you may know them from basic TCP programming under Unix/Linux and also from Windows. For more sophisticated applications you should consider the use of threads or of asynchronous socket IO. There is no protocol implemented - so don't expect that you can use this for complex applications.

Here is a sample test sequence using the same computer with two "black" windows:

     tcpReceiver 9514
     tcpSender "Hello E.Huckert" localhost 9514

File transfer with hu_ftp2: a larger socket based program

Practically all socket related test programs on the net are very simple - too simple to give you enough experience with the D/Phobos interface for sockets. I have therefor prepared a rather large file transfer program written in pure D with library Phobos. The advantage of this program: it runs under Windows and Linux, can be compiled on both operating systems and requires no installation. I have used dmd as compiler frontend. This program uses several TCP related classes plus an UDP based logger class. The zip file contains beside the D sources a Readme text file with some short explications and two make files: one for Windows and one for Linux (Debian and Ubuntu tested). Please note: the server is not multi-threaded and can thus accept only one connection.

Here is a picture illustrating the protocol used in program hu_ftp2:
 
               

Threads and synchronisation

As I write very often programs containing threads - in music applications or in communication applications - I need a working thread model. I C++ I used with success the Windows threads (see CreateThread() and EnterCriticalSection()) and under Linux the POSIXPhreads library (=POSIX standard 1003.1c-1995).

In D things seem a littler better as the language offers some keywords for synchronization like synchronized or shared. Threads are offered in the Phobos standard library. As usual the D documentation is not very clear - neither on synchronization nor on threads.

The main difference to other programming languages is that D uses thread local storage (TLS) where other languages use global storage So if you want to access global variables from threads you have to declare them as shared. This attribute seems to guarantee synchronized access which is essential for write operations. If you want to synchronize yourself the access to thread common variables then you should use the keyword __gshared.

My sample programs try to implement this picture:

               

The main program creates three threads. These threads try to increment a thread-external variable commonCount. As "increment" means: "write this variable" we need a synchronization mechanism. In many programming languages you have to use mutexes or semaphores or critical sections - constructs which are not always covered by the languages but rather by the operating systems or by APIs.

I demonstrate this in my sample programs threadsV1a.d and in threadsV1b.d. Both programs use the Thread object from import file core.thread. Mutexes (see the Mutex class in core.sync.mutex) are not needed for these simple applications. Whereas the sychronisation in the first sample program relies on the "shared" attribute the second sample program uses a slightly different approach: a synchronized code sequence in a member function called increment.

A very different thread model ist offered by import file std.concurrency: my sample program threadsV2.d is not based on thread objects, but on the spawn() call. The synchronization is implemented again by the "shared" attribute. Very important here: you must choose the right scheduler (I choose ThreadScheduler) if you want to have true thread IDs like in C/C++. The default scheduler (the docu doesn't mention this) seems to the the FiberScheduler. This second approach offers some benefits like message passing between threads (not used in the sample programs). I did not yet test this with newer Windows versions nor under Linux: may be that these operation systems use different schedulers.

 

Interfacing C modules and libraries on Linux

D has been designed for easy combination with C/C++ components. I is therfore particularly easy to use C components. C++ is a little bit more complex - the name mangling enters the play here. There are several D compiler backends (code generators) under Linux - among them the GNU gcc compiler. This compiler backend is used when you invoke the standard dmd compiler command under Linux. My samples are valid only for this GNU gcc base backend.

You can link with standard C object files as shown here. waveplay.o is the object file corresponding to the source file waveplay.c:  
 
      gcc -c -o waveplay.o wavefile.c  
      dmd -o xyz xyz.d waveplay.o  
 
If you need to use routines from a library built for Linux then the compiler option -L is your friend. In this examples the Linux library libasound will be used:  
 
      dmd -o xyz xyz.d -L-lasound  
 
The routines to be used in an external C module or in a Linux library must be declared as extern in the D program as in this samples:  
 
      extern (C) int wav_play(immutable char *pCStr);  
      extern (C) int snd_init();  
 
Special attention is necessary when passing D strings to C functions: the string concepts in both languages are different. But this is also known from the interaction between C and C++ (STL string class). In D an extra conversion to a zero-terminated byte array is necessary:  
 
      string fileName = „xyz.wav“;  
      immutable char *pCFileName = fileName.toStringz();  
 

This is illustrated in my complete sample program audioEH that consist of a D main program and C player module for WAVE files illustrating the call of ALSA routines under Linux. The compile and link instructions can be found in the comments of the source programs. The reason why I wrote that program is simple: as in most other high level programming languages there is no direct function to play WAVE files neither in the language D nor in its main library Phobos.


Contact

Copyright for all images, texts and software on this page: Dr. E. Huckert

If you want to contact me: this is my
mail address