SR Research Support Site

Graphics Programming using GDI

This section is intended for programmers who have at least some understanding of Windows programming. If you need an introduction, read the on-line help files for C++, about the Windows GDI. An excellent book for an introduction is "Programming Windows", Fifth Edition: by Charles Petzold, published by Microsoft Press (1998): chapters 4, 5, 14, and 17 are the most useful for programmers using this development kit. Although old, this book is still a good introduction for GDI programming in C under any version of Windows. The examples given in Experiment Templates Overview are intented for SDL programming.

The EyeLink Software Development kit (ELSDK) supplies several functions that simplify GDI programming and creating stimulus displays. As well, there are several C files used extensively in the templates that provide easy-to-use support for fonts, bitmaps, and images in your experiments. Where appropriate, these functions will be discussed in this section. This is a list of the C files that directly support graphics: you may wish to read through the code in these, or copy sections for use in your own graphics routines (don't modify the original files-keep them as a reference, and create renamed versions to modify).
w32_demo_window.c Typical window creation and support functions
w32_text_support.c Font creation and printf()-like text output
w32_bitmap_sppt.c Bitmap creation and display functions
w32_text_bitmap.c Formatted text pages; create text page bitmap
w32_freeimage_bitmap.c Load many types of image file to a bitmap

Graphics Modes

Windows supports many graphics modes, depending on your VGA card and driver. Using the "Display" item in the Windows Control Panel, you can explore and select the various modes available. Many video card drivers also supply a display-mode icon that is available at the right end of the desktop toolbar.

Resolutions and Colors

Graphics modes vary in resolution, number of colors available, and refresh rates. The most useful resolutions and color modes are listed below:

640x480Lowest common resolution, useful for pictures, simple graphics and single words printed in large text. Refresh rates over 160 Hz are usually only available in this mode.

800x600Moderate resolution, useful for pictures and pages of large text. Some monitors will support 160 Hz refresh rates in this mode.
1024x768High resolution, useful for pages of smaller text. Refresh rates over 120 Hz are not usually available

256 colorsIndexed color modes. You need to use palettes to draw graphics in this mode, which the templates do not currently do. This mode is most useful for simple graphics and text. Loaded images do not display well in this mode.

Hi color (15 or 15 bits)Displays real colors, but may show contour artifacts to images that have smooth transitions, as it only supports 32 levels of brightness.

True color (24 bits)Displays real colors, but may draw more slowly. Supports 256 levels of brightness (may actually be 64 levels on some video cards)

Drawing Speed

Drawing speed depends on the selected resolution and colors. In general, more colors, higher resolution and faster refresh rates all result in slower graphics drawing. For experiments that follow the templates, the time to copy a bitmap to the display is the most critical factor in selecting a mode. This should be tested for your card, by measuring the time before and after a bitmap copy. Ideally, the copy should take less than one refresh period so that the drawing is not visible, and so the subject sees the stimulus all at once. The systests application can be used to test the speed with which a full-screen bitmap can be copied to the display, but does not test other operations such as drawing lines, shapes, and text.

An important difference between Windows 2000/XP and DOS or Windows 95/98/Me is that graphics are not usually drawn immediately-instead, they are "batched" and drawn up to 16 milliseconds later. There are several ways to force Windows 2000 and XP to draw graphics immediately. The first is to call GdiFlush(), which forces any pending drawing operations to be performed. Batching can also be turned off by calling GdiSetBatchLimit(1), which turns off batching entirely (this has already been done for you in w32_demo_window.c). There are a few cases where batching might be useful, for example if a lot of small drawing operations need to be done together, or if a moving object needs to be erased and then redrawn at a different position as quickly as possible-batching could be enabled before drawing and disabled afterwards, as is done in w32_gcwindow.c for moving a gaze-contingent window.

Finally, many display drawing operations under Windows (especially copying bitmaps to the display) are done in hardware by the display card and the drawing function calls return immediately, long before the actual drawing is finished. While this can be an advantage if other non-drawing operations need to be done, this does make timing displays difficult. The function wait_for_drawing(HWND hwnd) is supplied for you, which both forces immediate drawing and does not return until all ongoing drawing is actually finished. The argument hwnd is a window handle, which can be either full_screen_window for the window created by w32_demo_window.c, or can be NULL to check for drawing operation anywhere on the display (including a second monitor if this is installed).

void CALLTYPE wait_for_drawing(HWND hwnd)
{
  HDC hdc;
  RECT rc;

  hdc = GetDC(validate_window(hwnd));
  if(IsWindow(hwnd))
    GetWindowRect(validate_window(hwnd), &rc);
  else
    {
      rc.top = dispinfo.top;
      rc.left = dispinfo.left;
    }

  GdiFlush();
  GetPixel(hdc, rc.top, rc.left);
  ReleaseDC(validate_window(hwnd), hdc);
}

The code for this function is very simple: it calls GdiFlush(), then calls GetPixel() to read a single pixel from the display. Because Windows cannot determine what color a pixel will be until drawing is finished, this read does not return until drawing is completed. As you can see, most of the code is involved with preparing for the call to GetPixel(), so if you are writing your own graphics code it may be more efficient to simply call GetPixel() following your own graphics code.

Display Mode Information

As part of initializing your experiment, you should check that the current display mode is appropriate to your experiment. For example, a fast refresh rate may be required, or a resolution that matches that of a picture may be needed. An experiment should always be run in the same display mode for all subjects, to prevent small differences in appearance or readability of text which could affect subject performance.

Information on the current display mode can be measured by calling get_display_information(). This fills a DISPLAYINFO structure with the display resolution, colors, and the refresh rate. If you use the global DISPLAYINFO structure dispinfo, then the macros SCRWIDTH and SCRHEIGHT can be used to compute the screen width and height in pixels.

This is the definition of the DISPLAYINFO structure:

  typedef struct {
     INT32 left;      // left of display
     INT32 top;       // top of display
     INT32 right;     // right of display
     INT32 bottom;    // bottom of display
     INT32 width;     // width of display
     INT32 height;    // height of display
     INT32 bits;      // bits per pixel
     INT32 palsize;   // total entries in palette (0 if not indexed)
     INT32 palrsvd;   // number of static entries in palette
     INT32 pages;     // pages supported
     float refresh;   // refresh rate in Hz (<40 if refresh sync not available)
     INT32 winnt;     // Windows: 0=9x/Me, 1=NT, 2=2000, 3=XP
  } DISPLAYINFO;

Several fields in this structure require further explanation. The palsize field will be 0 if in 32, 24 or 16-bit color modes, which are required for image file display. If this is nonzero, you are in 256-color mode (or even 16-color) mode, which is only useful for drawing simple graphics. The pages field is always 1, as multiple pages are not supported. The refresh field is the measured display refresh rate, which may differ from that reported by the operating system. If this is less than 40, the wait_for_video_refresh() function is not working (probably due to an incorrect implementation of VGA registers on the video card). In this case, the system's refresh rate can be retrieved with the following code:

      refresh = (float)GetDeviceCaps(NULL,VREFRESH);

Finally, the winnt field indicates what version of Windows the application is running under. This can distinguish between Windows 95/98/Me (where realtime mode is ineffective), Windows NT (for which realtime mode will stop the network from working), Windows 2000 (realtime mode works and does the keyboard), and Windows XP (realtime mode works but disables the keyboard).

Adapting to Display Resolutions

When the subject-to-display distance is twice the display width (the recommended distance for EyeLink operation), then the display will be $30^{\circ}$ wide and $22.5^{\circ}$ high. This allows you to compute pixel sizes of objects in degrees, as a fraction of display width and height. The templates also use this method to adapt to different display resolutions. Complete independence of display resolution is not possible, as fonts and small objects (i.e. the calibration targets) will look different at low resolutions.

Synchronization to Display Refresh

The image on the phosphor of the display monitor is redrawn from the video memory, proceeding from top to bottom in about (800/refresh rate) milliseconds. This causes changes in graphics to appear only at fixed intervals, when the display refresh process transfers that part of the screen to the monitor. Graphics at the top of the screen will appear about 0.5-2 millisecond after refresh begins, and those at the bottom will appear later.

The simplest way to determine just when graphics will be displayed to the subject is to pause before drawing until the refresh of the display begins, using this function wait_for_video_refresh(). This function returns only after video retrace begins; if it is called while video retrace is active, it will continue to wait for a full retrace period, until the retrace begins again. This is designed to provide the maximum time for drawing to the display. If you just want to check to see if retrace is active, call in_vertical_retrace() instead. However, be sure to allow enough time between calls to this function, as vertical retrace can be active for up to 4 milliseconds, and this could be interpreted as multiple video refresh intervals passing.

In summary, if graphics can be drawn quickly enough after the display retrace, they will become visible at a known delay that depends on their vertical position on the monitor. Placing a message in the EDF file after drawing which includes the delay from the vertical refresh allows applications to precisely calculate the onset time of a stimulus.

Full-Screen Window

Every Windows program must have a window. The module w32_demo_window.c implements the minimum functionality needed for supporting the full-screen window needed for experiments. It includes functions to create and destroy the window, and to clear it to any color. Also included is a function to process messages for the window.

Creating the Window

The function make_full_screen_window() creates the experiment window. It first registers a window class, which specifies the defaults for our window. It then creates a new window with no border and sized to fill the entire screen. It then makes the window visible, and waits for it to be drawn. Since Windows sometimes erases other windows before drawing our window for the first time, we can't draw to our window until it's seen its first WM_PAINT message, or our graphics could be erased by the redrawing of the window. The code in this function is very critical, and should not be changed. After creating the window, it pauses for 500 milliseconds while running the message pump, which allows Windows to remove the taskbar from the desktop.

This functions also calls GdiSetBatchLimit(1), which forces graphics to be drawn immediately by Windows, instead of being deferred until a later time. This does not guarantee that some drawing operations (especially bitmap copies) will be finished before a drawing function returns-use wait_for_drawing() after drawing to ensure this.

Clearing the Display

The window (and the whole display) can be cleared by calling clear_full_screen_window() and specifying a color. Windows colors are specified by the red, green, and blue components, using the RGB() macro to combine these into a COLORREF number.

Processing Windows Messages

All Windows messages for the window are handled by full_screen_window_proc(). This handles key press messages by passing them to the ELSDK library function process_key_messages(), which passes the keystrokes to any library functions that may be running, such as do_tracker_setup(), or saves them for later retrieval by getkey(). It also intercepts messages that Windows send to close the window, and calls terminal_break(1) to inform your experiment. It makes the mouse cursor invisible when only our window is visible, so it doesn't interfere with the experiment displays. Finally, it clears parts of the window to target_background_color which were covered by dialog boxes or other windows. Any display information in these areas is lost, so you may want to add redrawing calls here, or save the entire screen to a bitmap before calling up a dialog box, then restore it afterwards.

Changing the Window Template

Usually, there is little reason to change the code in w32_demo_window.c, unless you are adding functionality to the window. For example, you might add palette support when clearing the window. The handling of messages can be changed, for example to make the mouse cursor visible.

Registering the Window

Your full-screen window must be registered with the eyelink_gdi_graphics library in order to perform calibration, drift correction, and to display camera images. This is done by calling init_expt_graphics(). This allows ELSDK to be used with languages such as Visual Basic, that must create their own windows. During calls to do_tracker_setup() or do_drift_correct(), ELSDK will intercept some messages to your window, and will draw calibration targets and camera images into it. You should call close_expt_graphics() to unregister the window before closing it.

Windows Graphics Fundamentals

The great advantage to creating graphics in Windows is that there are powerful drawing tools such as TrueType fonts, patterns, pens and brushes available, and these work in any display mode without changes. However, this power comes at the cost of adding setup code before drawing, and cleanup code afterwards. Also, the Windows programming interface for some graphics operations can be rather complex, so you'll have to become familiar with the help files (or Visual Studio InfoViewer topics) for the Windows API.

The following discussion only covers programming issues that need to be considered for experiments. You should study the Windows API documentation that came with Visual Studio on the GDI for more general information, or consult a good book on the subject for more information.

Display Contexts

Drawing in Windows must always be performed using a display context (DC). This allows drawing to the full screen, a window, or a bitmap. Usually display contexts are allocated as needed and released immediately with GetDC(full_screen_window) and ReleaseDC(full_screen_window).

Each time you allocate a DC, you will have to select new objects such as brushes, pens, palettes and fonts into the DC, then reselect the original objects before releasing the DC. You could also select stock objects (such as white brushes, black pens, and the system font) instead of restoring the old objects.

When you allocate a DC, you need to supply a window handle (HWND). If this is NULL (0) you will get a DC that allows writing to the whole screen. It's better to use the full-screen window handle (usually full_screen_window) that your experiment created, however.

Specifying Colors

Windows colors are specified by the red, green, and blue components, using the RGB() macro to combine these into a COLORREF number. By combining this value with 0x02000000, (for example, RGB(0,255,0)|0x02000000) the color will be drawn as a solid shade. Otherwise, colors might be drawn using a dot pattern in 256-color modes.

Pens and Brushes

The drawing of lines is controlled by the currently selected pen. Filling in shapes is controlled by the currently selected brush. Both pens and brushes can be created in any color, or made invisible by selecting the stock NULL_PEN or NULL_BRUSH.

You must remember to remove pens and brushes from the DC before disposing of it. You can do this by reselecting the original pens or brush, or by selecting a stock pen or brush. Created pens or brushes must be deleted after being deselected. For an example, see the file w32_playback_trial.c from the eyedata sample experiment.

Fonts and Text

Windows has many options for selecting and using fonts. The most important text requirements for experiments are selecting standard fonts, printing of short text messages and words, and drawing of pages of text for reading trials and instruction screens. These are implemented in the template support modules w32_text_support.c and w32_text_bitmap.c.

Fonts are created by calling get_new_font(), supplying a font name, its size (height of a line in pixels), and whether the font should be boldface. The font created is stored in the global variable current_font, and is not deleted until a new font is selected. The most useful standard font names are:

Arial.png
A simple, proportionally spaced font.
courier.png
Monospaced font. Rather wide characters, so less will fit per line.
times.png
A proportionally spaced font, optimal for reading.
Fonts can be drawn with directly by selecting current_font into a display context, and released after drawing by selecting the system font, as in this example:

if(current_font) SelectObject(hdc, current_font);
SetTextColor(hdc, text_color | 0x02000000L);
TextOut(hdc, x, y, text, strlen(text));
SelectObject(hdc, GetStockObject(SYSTEM_FONT));

Drawing of fonts may be slower when a character is drawn for the first time in a new font. You can increase drawing speed by first drawing the text invisibly (in the background color), which will cause windows to create and cache a bitmap for each character. These cached characters will be used when you redraw the font in the foreground color.

When there is a need to print a few characters or a line of text, the function graphic_printf() can be used. You can specify a foreground and an optional background color, a position for the text, and whether to center it. The text is generated using a format string similar to the C printf() function.

Multi-line pages of text can be printed using draw_text_box(), supplying the margins and a line spacing in pixels. Optionally, boxes will be drawn at the position of each word on the EyeLink tracker's display, to serve as a reference for the gaze cursor during recording. You need to supply a display context to this function, which allows this function to draw to either the display or a bitmap. A bitmap containing a page of text can be created and drawn in one step with text_bitmap().

Drawing With Bitmaps

Drawing graphics takes time, usually more than one display refresh period unless the graphics are very simple. The progressive drawing of complex stimuli directly to the display will almost certainly be visible. This is distracting, and makes it difficult to determine reaction-time latencies. One method to make the display appear more rapidly is to draw the graphics to a bitmap (a memory buffer), then to copy the bitmap to the display. Bitmap copies in Windows are highly optimized, and for most video cards and modes the entire display can be updated in one refresh period.

The type of bitmaps that need to be used for drawing are device-dependent bitmaps (DDB). The Windows documentation is full of routines that support device-independent bitmaps (DIBs), but these are actually only used for loading and saving resources, and for copying images on the clipboard. Don't use any DIB functions with bitmaps: they will not work reliably, and are often slower.

Drawing to Bitmaps

The C source file w32_bitmap_sppt.c (used in the picture and text templates) contains functions to copy bitmaps to the display, and blank_bitmap(), a simple function to create and clear a bitmap that is the same size as the display. Its code is used in the discussion below.

Before accessing a bitmap, we have to create a memory device context (MDC) to draw in. We first get a DC to the display, then create a memory device context compatible with the display DC. We can then create a bitmap, in this case of the same dimensions as the display. We could also make a smaller bitmap, which would save memory when several bitmaps are required.

Then, the bitmap is selected into the MDC. We save the handle of whatever bitmap was originally selected into the DC, to use when deselecting our bitmap later.

  hdc = GetDC(NULL);
  mdc = CreateCompatibleDC(hdc);     // create display-compatible memory context
  hbm = CreateCompatibleBitmap(hdc, SCRWIDTH, SCRHEIGHT);
  obm = SelectObject(mdc, hbm);      // create DDB bitmap, select into context

Now, we can draw it to the bitmap using the same GDI functions as would be used to draw to the display. In this case, we simply clear the bitmap to a solid color:

  oBrush = SelectObject(mdc, CreateSolidBrush(bgcolor | 0x02000000L));
  PatBlt(mdc, 0, 0, SCRWIDTH, SCRHEIGHT, PATCOPY);
  DeleteObject(SelectObject(mdc, oBrush));

Finally, we select the old bitmap to release our bitmap, and clean up by deleting the MDC and releasing the DC:

  SelectBitmap(mdc, obm);
  DeleteDC(mdc);
  ReleaseDC(NULL, hdc);

We now have our bitmap, which we can draw in by creating an MDC and selecting it. Remember to delete the bitmap with DeleteObject() when it is no longer needed.

Other functions in the template modules that draw to bitmaps are text_bitmap() in w32_text_sppt.c, draw_grid_to_bitmap() in w32_grid_bitmap.c, and bmp_file_bitmap() in w32_bmp_bitmap.c.

Loading Pictures

The function image_file_bitmap() defined in w32_bmp_bitmap.c loads an image file from disk, and creates a bitmap from it. It can also resize the image to fill the display if required. It uses the freeware "FreeImage" library for this purpose (www.6ixsoft.com). This loads many picture formats, including BMP, PCX and JPG (but not GIF). JPG files are smallest, but load more slowly and may have artifacts with sharp-edged images such as text. Creating a 256-color PCX file is much more efficient for text or simple line drawings. This library always loads images as an RGB bitmap, so any palette information in the original image is lost. This means that even 2-color images will be displayed in garish, saturated colors in 256-color modes, and programs that use loaded images should always run in 16, 24, or 32-bit color display modes.

The arguments to this function are image_file_bitmap(fname, keepsize, dx, dy, bgcolor). This function creates and returns a bitmap (NULL if an error occurred) that is either the size of the image (keepsize=1) or full-screen sized. If a full-screen bitmap is created, the bitmap is cleared to bgcolor and the image in copied to the center of the bitmap, resized to (dx, dy), or left its original size if dx is 0. Resizing is done using StretchBlt(), which can be rather slow, and the results will be poor if the image contains text or other sharp detail. It is best to create images of the same dimensions as the display mode you will use for our experiment.

Copying to the Display

Copying the bitmap to the display is similar to drawing: create a memory device context for the bitmap, get a DC for the display, and use BitBlt() to do the copying. It's a good idea to call GdiFlush() to ensure that drawing starts immediately. The Windows GDI passes information to the VGA card's driver, which usually does the copying in hardware. This means the call to BitBlt() may return long before the drawing is done-if this is not desirable, call wait_for_drawing() or call GetPixel() for a point on the display. Both functions will return only once the drawing is completed.

The w32_bitmap_sppt.c source code file is used in most of the sample experiment templates, and has two functions to copy bitmaps to the display. To copy an entire bitmap to the display, call display_bitmap(), specifying the position at which to place the top left corner of the bitmap. A rectangular section of a bitmap can be copied using the display_rect_bitmap() function.

All the source code examples create a full-screen bitmap to the display. It is obviously faster to create smaller bitmaps and copy these to the center of the display instead, but this should only be needed for slow video cards. For example, the margins of a page of text usually are $2^{\circ}$ from the edges of the display. Copying only the area of the bitmaps within the margins will reduce the copying time by 40%.


Copyright ©2006, SR Research Ltd.