EE4273 Lab Project Stephen Manley - 239585

Objective

The objective of this project is to design and develop several data acquisition and modification systems and to develop an experimental methodology to test these designs on their merits.

Equipment and Technical Requirements

The data acquisition system needs to work with a DT2821 board as provided by the UNB Electrical Engineering department. The software as developed is to have several separate processes that handle the collection and processing of data.

All software is to compile on the QNX operating system in the GD-120 computer engineering lab. VEDIT and Watcom C are available for debugging tools. The source code must make use of process forking features available in the operating system. At least two of the possible multi-process synchronization techniques available in QNX need to be used, for example, message passing and shared memory. Bonus marks will be provided for the implementation of message passing, message queues and shared memory implementations.

General Design Overview

The software is to use two main divisions into reader and collection processes. The specifications for these are as follows:

Reader Processes

These are processes (minimum two) that are to use the QNX operating system features to read data that has been acquired by the system. They should be able to execute anywhere on the network (e.g.. Remotely). Once these reader processes have obtained the data, they should preform some arbitrary manipulation on, or based on, the data and then send this information to a digital to analog converter for display on an oscilloscope.

The methodology that is used to receive the data from the ADC will vary depending on the solution that is implemented (see comments below).

Collector Process

These are collector processes that gather the data from the analog to digital board. They are responsible for initializing the ADC and managing data collection. How this information is distributed and the flow of data managed is left as part of the design solution implemented for each possible solution.

In order to calculate the process execution time, your software should use the clock() function and some math to determine the maximum throughput rate.

Solution A - Shared Memory (Semaphores)

This solution makes use of the supplied features for shared memory and data structures in QNX. Mutual exclusion is guaranteed through the use of semaphores within these structures. The effect is that many processes can make use of and change information stored in the same memory location without fears of corrupting the data.

Process Communication

The functions new_queue(), queue_get(), and queue_put() are used to create and access the shared memory pool. Within this shared memory structure, there are three semaphores that manage access:

navail:	Updated when there is data available in the queue

nspace:	Indicates amount of free space available in the queue

lock:	Manages mutual exclusion when changing data

Data Structures

Two of these queues are used in the program, one for each of the reader processes that are implemented. This is done to make sure that each process receives an identical set of data points from the ADC, useful when debugging and comparing the outputs.

A QNX proxy is used to set the update of the ADC data according to a timer interrupt. This proxy guarantees synchronization of the data gathering routines. This is set up via the qnx_hint_attach function.

Data Flow

The process is initially run, sets up the two new queues to share memory and then it forks off two additional reader processes, READERA and READERB. The reader methods accept a queue pointer to talk to and a number of cycles to run. The running time is determined from the time it takes for this process method to complete, calculated on a per cycle basis.

The readers execute a queue_get, obtain information, manipulate it and then send the information to the dac. This operation is repeated for the number of cycles passed into the method.

The collector process blocks until it receives a message from the proxy then gets data from the ADC. This information is then put into the two queues with the queue_put() method. The time that this operation takes is then calculated and averaged with the clock() function.

Debugging Notes

Shared memory structures are simple to implement and work well. This is important to note; Shared memory was the quickest to implement and the easiest to debug. It is also important that the timing between the processes be correct and that the naming conventions be standard; Also, maintaining track of pointers is important. A major debugging problem occurred when the names of the two queues were mixed up and the pointers were accidentally assigned to be the same.

Experimental Results

The experimental results list the time taken for the completion of the associated process for an average read/write cycle. The read/write cycle is defined as the time it takes for one complete acquire-send or receive-send cycle. This is determined from the system clock() function with times taken before and after the method is started. The average time is then calculated by removing the timer interrupt delay (50ms), then dividing by the number of clocks per ms and the number of cycles that occurred.

It is worthy to note that there is a great deal of tweaking that can be done in system performance measuring, or "code profiling" that is beyond the scope of this lab assignment.

Experimental Results Table - Shared Memory - Node 3
Process Name	Test 1	Test 2	Test 3
Reader A	43.800ms	43.945ms	43.945ms
Reader B	43.800ms	43.945ms	43.945ms
Collector	43.800ms	43.945ms	43.945ms

Solution B - Message Passing with Direct Naming

This solution makes use of message passing for interprocess communication and synchronization. This makes use of message blocking and event handling to handle the reading and distribution of information within the program.

Process Communication

The QNX operating system features that are used are Send(), Receive(), and Reply(). All data transfer occurs through the system message calls and no external data structures for process communication were implemented.

In order to identify the processes, direct naming conventions were used. The QNX calls qnx_name_attach() were used to define global names for the processes running on all machines. Once this name has been registered, a process can then be found using the qnx_name_locate() method.

Data Structures

A QNX proxy is used to set the update of the ADC data according to a timer interrupt. This proxy guarantees synchronization of the data gathering routines. This is set up via the qnx_hint_attach function.

Messages are simple characters that are passed around the various processes and handled in the collector. 'A' identifies Reader A, 'P' identifies the collector proxy, etc.

Data Flow

Once the readers are forked, the collect process begins. Since the event handler is in the collect process, it acts as the coordinator for all data transfers. The proxy spawns a messageID that tells the collector to grab a new sample from the ADC and store it for access by the readers.

When a message request for data is made by the reader, it sends a message requesting the information to the collection process. This send is then blocked, so that the reader will wait until it receives a reply from the collector. Once this send request is made to the collector, it enters the event handler loop. The collector will the reply with the current sample to the reader process. Each reader is synchronized to wait a minimum of 25ms for a new sample to be taken, however, there is no reason that this needs to take place. It is only done to help eliminate duplicate samples from causing problems with debugging efforts.

Debugging Notes

Messages proved very challenging to debug properly. The hints must be started after the process has had a name attached, and the proxy needs to be started before the process has a name. Synchronization issues with the readers were difficult to deal with as well; The timings need to be equal in both of the reading processes to keep the values consistent.

Other problems were caused in naming the processes; It was important to first query QNX to determine that there was no existing process with the same name, for instance. This demonstrates the importance of checking for error conditions in multitasking real time operating systems.

Experimental Results

The experimental results list the time taken for the completion of the associated process for an average read/write cycle. The read/write cycle is defined as the time it takes for one complete acquire-send or receive-send cycle. This is determined from the system clock() function with times taken before and after the method is started. The average time is then calculated by removing the timer interrupt delay (50ms), then dividing by the number of clocks per ms and the number of cycles that occurred.

It is worthy to note that there is a great deal of tweaking that can be done in system performance measuring, or "code profiling" that is beyond the scope of this lab assignment.

Experimental Results Table - Message Passing - Node 3
Process Name	Test 1	Test 2	Test 3
Reader A	11.951ms	10.273ms	10.267ms
Reader B	11.951ms	10.278ms	10.275ms
Collector	45.872ms	45.872ms	45.797ms

Solution C - Message Queues

This solution makes use of QNX message queues for interprocess communication and synchronization. This approach involves placing a message on a queue and then managing who is allowed to take it off through blocking and other synchronization techniques.

Process Communication

Process communication is handled through the new_queue(), mq_send() and mq_recieve() QNX methods.

Data Structures

Two message queues are used in the program, one for each of the reader processes that are implemented. This is done to make sure that each process receives an identical set of data points from the ADC, useful when debugging and comparing the outputs. There are no built in protections for the accessing as well, so using the two queues with blocking aids eliminating interference from unsynchronized processes.

A QNX proxy is used to set the update of the ADC data according to a timer interrupt. This proxy guarantees synchronization of the data gathering routines. This is set up via the qnx_hint_attach function.

Data Flow

Once the program starts, the two readers are forked off. Once this has happened, the collect process starts up. It blocks for a timer interrupt using the QNX Receive() function, then proceeds to get a word from the ADC. Once it has obtained valid data, this information is sent to the message queues for each of the readers.

When the readers are started, they will immediately block until their respective message queues obtain information from the collect process. Once this has occurred, the information is sent to the DAC and the process repeats.

The blocking design guarantees that there will never be more than one message in the queue, and that the all data collected will be handled by the readers. In addition, the message queues are priority driven. It is important that the collection and receiving queues have the same execution priority in this example for maximum throughput.

Debugging Notes

Message queues, like shared memory are simple to implement and work well. The implementation of these took little time compared to the message passing alternative, and the implementation worked with little tweaking.

It is important to not that the messages will be buffered to disk. During debugging, when a process goes astray or zombie, it will often leave behind large message buffers that need to be manually cleared from the disk. Not doing so can quickly fill up even a large hard drive and present a difficult-to-locate source of error.

Experimental Results

The experimental results list the time taken for the completion of the associated process for an average read/write cycle. The read/write cycle is defined as the time it takes for one complete acquire-send or receive-send cycle. This is determined from the system clock() function with times taken before and after the method is started. The average time is then calculated by removing the timer interrupt delay (50ms), then dividing by the number of clocks per ms and the number of cycles that occurred.

It is worthy to note that there is a great deal of tweaking that can be done in system performance measuring, or "code profiling" that is beyond the scope of this lab assignment.

Experimental Results Table - Message Queues - Node 3
Process Name	Test 1	Test 2	Test 3
Reader A	44.160ms	44.036ms	44.036ms
Reader B	44.160ms	44.036ms	44.036ms
Collector	44.160ms	44.036ms	44.036ms

Conclusions

The environments that each of the synchronization and resource sharing techniques were tested in were kept as equal as possible. The experimental results were quite clear; Message queues and shared memory were very quick to implement, however their results were rather lackluster compared to the very quick read times offered by the message passing with direct naming alternative. Everything comes at a price - the message passing was the most difficult of the two to get working, however.

The message passing offered approximately a 400% increase in throughput - a very significant figure.

Operating system activity also played a role in the test, with variations in how the memory was allocated in the system having an effect on the overall performance of the system. The tests were executed within the Photon GUI, which had a positive effect on the run times for an unknown reason (all times gained approximately 0.5-0.8ms within the environment).

All three designs scaled very well, with the number of cycles from 1 to 1000 being tested. There was little or no difference in the mean times measured. The time involved in testing larger numbers was too great to be practical given the scope of this lab. As the number of samples becomes very large, message passing looks more attractive and should begin to share a significant advantage over the other techniques due to the smaller memory footprint - messages do not need to be stored for any length of time. In addition, the data is passed directly in the message - saving even more time.