GNSS Flight recorders

General
FAQ (6 Oct 08)
Home (6 Oct 08)

Manufacturers
CIMA Approved (6 Oct 08)
GAC Approved (29 Jul 02)
IGC Approved (28 Jul 02)

Software
Specification for FR software (15 Jan 09)
Specification for FR software.doc (77k) (15 Jan 09)
MLR data transfer (6 Oct 08)
MLR firmware (6 Oct 08)
MLR Flight Recorder instructions.pdf (107k) (1 Aug 05)
MicroFLAP (27 Jul 05)

spare
2003 WMC reliability analysis (5 Sep 03)

Standard
About the standard (6 Oct 08)

Utilities
microFLAPload (5 Jun 09)
File conversions (6 Oct 08)

Utilities OLD
File Conversions (20 May 09)
Showfile (26 Oct 05)

Contact rmh

Software: Specification for FR software
Updated 15 Jan 2009 Specification for Microlight and Paramotor championship GPS track data collection software

Specification for Microlight and Paramotor championship GPS track data collection software.

 

There is an urgent need for a new source of GPS flight data recorders for use in National and International Microlight and Paramotor competitions.  This specification explains that the problem is not one of hardware, but the lack of software to download and convert the recorded data into a useful form, and it lays out the requirement needed to achieve this.

 

Persons interested in helping to build this software should get in touch with  Richard Meredith-Hardy

 

version 1     15 Jan 2008

Background

The competition scene in microlights and paramotors is quite active.  European and World championships have been occurring under the auspices of CIMA, the microlight commission of the FAI in alternate years since 1985 and 1997 in each sport respectively.  National teams for international championships are usually selected by performance in National championships or championship leagues.  Today, one typically expects around 60 aircraft from 12 nations in a World Microlight championship, and 100 – 120 from 18 nations in a World Paramotor Championship.

 

An international championship lasts one week, and depending on the weather, pilots will be expected to fly between 6 and 14 tasks.  Tasks are divided into three categories; Precision tasks, which are usually flown local to the airfield, and Cross-country Economy or Navigation tasks which are flown away from the airfield.

 

The way a pilot performance in a task is scored is specific to the task, and is usually rather a complicated formula.  A score is arrived at via a three stage process:  Data collection, flight analysis, and scoring. 

 

Since 2003, the pilot performance in nearly all cross-country flights away from the airfield has been verified by GPS track data.  Thus most of the data collection is inherently electronic, but it has to be downloaded from the GPS device, analysed, typically in a specialist program like microFLAP, and the key elements according to the task passed to the scoring, typically in a spreadsheet like Excel.

 

The way this is done is subject to some particular requirements.  Most notable is that although a GPS is on board recording the flight, the pilot is expected to plan and fly all tasks by ‘traditional’ means and must not have access to any GPS flight information whilst undertaking the task.

 

It was appreciated at an early stage that a reliable and fair system would need quite a rigid specification for both the GPS itself and the data it produced.  These rules are maintained in FAI Section 10 Annex 6.

 

Whilst the gliding community had been using loggers since the mid-1990’s, they were specialist devices, very expensive, and generally displayed at least some navigational information.  In practice, the rules for microlights and Paramotors were originally written around the first device to become available which was reasonably priced and more-or-less met the basic requirement.  This was the MLR SP24-XC which is a normal hand-held GPS with firmware modified by the manufacturer so it only displays GPS time, number of satellites in view, and battery and memory status.  There are two firmware variants; 2 sec logging and 5 sec logging, which with a memory for 8000 fixes gives an endurance of 4h 26 min or 11h 6 min.

 

The talented Spaniard, Jose Luis Esteban designed and wrote a MLR download software which has been demonstrated many times to be an essential part of the championship organizer’s armoury.

 

The MLR has some significant faults, but in general has proven to be robust and reliable.  Unfortunately however, not long after it came into use, MLR were taken over by another company who discontinued the SP24-XC, and today it is impossible to find. Only one other type, made by Air Observer in South Africa, has become available since 2003 but it is not very cheap. It is the responsibility of teams or pilots to supply their own loggers, but with such limited availability they often don’t have them. Championships have been saved more than once by FFPlUM, the French microlight federation, who bought a substantial quantity of MLR’s and often lend their spare ones to pilots. 

 

With so many cheap GPS loggers available today, you might be wondering why there aren’t other types in use.  The answer is quite simple; the hardware requirement in the specification is only half the story.  The other half is in reliably and quickly getting the data out of them in a useful format.

 

If you have 120 pilots in a championship, each with a primary and a backup secondary logger, and the pilots fly two tasks in a day, it means the organization has a significant job just to download these 480 tracks from the loggers, let alone analyse and score them as well.  Practice has shown that the downloading procedure must be as automated as possible because with so many tracks to handle, manual intervention is an open invitation for error.  Even one simple error such as the loss of a track can lead to the voiding of the whole task, and if it is a poor weather week, the loss of one task can lead to the loss of the whole championship.  Even with the automated procedures we currently have, more than one championship has come close to disaster through inadvertent loss of track data.

 

Both the MLR and the Air Observer have had software written for them which meets the specification in FAI Section 10 Annex 6.  In principle, this software:

1)     Downloads all the data from the GPS and saves it to the host computer.

2)     Identifies from the ID of the GPS who that track belongs to.

3)     Translates the data to the required format (a slight variant of .igc format)

4)     Writes it to disk with the correct file name.

 

In practice, both these softwares do a bit more than this so it is easier to analyse, see the notes in Annex 1 below.

 

An approval system is described in FAI Section 10 Annex 6 which permits the use of these loggers in championships. Approval is inclusive of the download software, which perhaps explains why there are so few approved models. 

 

On the face of it, the approval requirements may seem over-demanding, but in 2005, the World Microlight championships and the World Paramotor Championships were held simultaneously at Levroux in France.  In this case, even with the generosity of FFPlUM, there were not enough MLRs for everyone, and pilots without an approved type were permitted to use a limited selection of Garmins sealed in an opaque container.  In the Paramotor championship it usually took about the same time to manually get the data from 7 Garmins into a useful form as it took to get it from the 150 MLR’s and Air Observers with their automated software. 

 

Levroux unequivocally demonstrated the case for specialist software, but the demise of the MLR demonstrated that creating it could be a lot of work for nothing.

Alternative loggers.

Some cheap loggers have been investigated, eg the DG 100 but they have one important fault, no serial number is delivered with the data.  This makes them impossible to identify automatically, thus introducing an unacceptable level of risk of error into the data downloading procedure.  These types of data loggers also tend to come with their own USB drivers which are likely to be difficult and unreliable to hack into with a bespoke data transfer software.  And of course in this rapidly changing world, they could be discontinued by the manufacturer at any time just like the MLR was.

 

Another solution are the new breed of devices which transmit GPS track data to a server via GPRS eg the Sinotrack.  This is highly attractive as it offers the possibility of a pilot being scored before he actually returns back to the airfield.  However, in many countries there are legal implications in using these devices from the air, and they have been shown in some countries to work quite poorly from the air; this is thought to be something to do with the configuration of the ground antennas.  A solution would be to buffer unsent track fixes in local memory, and some are capable of this, but usually they’re not intended to be recording fixes at a rate of more than one or two per minute, and we require one fix per second, so they simply don’t have the memory capacity to save track fixes for very long at this rate.

A new way forwards.

Recently, a new genre of loggers have begun to appear which are sometimes described as ‘photo loggers’.  They are intended to be used to record a fix every second whilst you are out and about taking photographs, and then when you get home, a special software is used to match the times in the track with the times of the photos to effectively ‘geolocate’ the photos you took. 

 

The point about these devices is that they often record their tracks to a SD card, or to on-board memory which in either case appears to the host computer as just another hard drive in exactly the same way as a standard memory stick does.

 

Some examples of these devices are: Geochron, Photo-GPS and Amod AGL3080

 

For our use, these offer several significant advantages:

1)     No special driver software.

2)     Large or effectively unlimited memory capacity.

3)     The possibility to deliberately write an ID and other information to the device which electronically identifies it (or at least the SD card) to the download software.

4)     The same download software can be used for current and future devices of this same generic type, so it is basically quite a future-proof concept.

 

Of course each specific device is likely to write its log files to its memory in a different file name protocol and directory structure, and the track data itself is likely to be slightly different (usually variants of NMEA).  But in the context of a generic downloader software, this could be easily managed with a simple configuration file for each specific type of device.  The basic business of connecting and transferring the raw data is the same for all of them.  They also tend to be platform independent, ie they are seen as a hard drive from a Mac or Linux just as easily as they are from Windows.

The software specification

General principles

  • The name of the application is yet to be decided.  (or the configuration xml file suffix, which for the time being is referred to as .xml below).
  • Although the author(s) may take credit, it is an open-source project released under the GNU general public licence.
  • Simple to install on the host computer.
  • Platform independence is preferred, ie Java, but the software should at least work on MS Windows 2000/XP/Vista.
  • The language is English, UTF-8  (optional possibility of future language add-on packs).
  • It should be as simple as possible to operate, with an intuitive user interface.
  • The user should be continuously informed of the current status (ie waiting for data connection, downloading, analysing, final file saved, please disconnect device – or error, with some useful information of what the error is).
  • Checks should be made for data corruption.
  • Data should generally never be overwritten or deleted.
  • It should be appreciated that rather large volumes of data might be encountered; 2 Gb SD cards are not unusual.
  • It should be as fast as possible in operation (200 tracks @ 1 minute each = 3h 20 min which is a long time).
  • It should be multi-tasking (ie separate instances of the software could be downloading different devices in different slots simultaneously).
  • Configuration files in standard XML.

Modes of operation outline

1)     Settings
This is where general program settings are made.

a.      Paths to where add-in files are located (master device configuration files and language files).

 

2)     Championship setup.
This is where championship settings are made.

a.      The name of the championship.

b.      General time offset.

c.      Operator language.

d.      Organizer password (see below).

e.      Path to where all backup files are saved.

f.       Path to where the final .igc data is saved.

g.      Other key information pertaining to the whole event.

3)     Device setup.
This is where the device (or memory card) is configured for the championship so it can be instantly identified every time it is connected to a host in future.

a.      Operator selects the device type from a list of options (ie master device configuration files).

b.      Operator enters the pilot name, competition class, competition number, whether this is the pilot’s primary logger or a backup, and any other items required by the master device configuration file.

c.      Option to completely clear the device memory.

d.      Software writes a local device configuration file to the device, and optionally (see more below) the device setup file. 

e.      This is the only time the software writes a new file to the device memory so clear warnings must be given before any action is taken.

 

4)     Task setup.
This is where the settings for the current task are entered.

a.      Task number

b.      Task ‘filter’ time window begin and end.

 

5)     Data download.
This is the most common mode and where all the information from the other four modes is put to use.

a.      The software recognizes a new source has been inserted in the slot/USB socket. 

b.      Identifies the device, and downloads all new data from it to disk. 

c.      The data is then analysed and all track points occurring within the filter window are collected, sorted in chronological order and converted to .igc format. 

d.      This data is written to a file in the correct location with the correct file name. 

e.      This whole sequence should occur without any intervention apart from the physical act of the operator actually plugging the device into the host.

Further notes

Settings files

Except for the device setup file which is in a format specific to the device, these are all XML files.  The first attribute in all these files is a descriptor of the type of configuration file it is.  Where necessary, attributes from one file may be copied to another.

 

The master device configuration file

This is a file published by CIMA which lays down the ‘approved’ specification of each separate device.  The file lives in a library connected to the program and contains the required settings for the device. 

 

In principle then, if an unidentified device is connected to the host, the program goes to the Device setup page where the operator can choose exactly which device the hardware is from the list of master device configuration files.  When one is selected, then the operator may be asked further questions about how the hardware should be configured, depending on the contents of that master device configuration file. 

 

The championship configuration file

This file lives in the root of the directory where all backup files are saved (as set on the championship setup page) and is named by the operator; typically something like championship_name.xml  It contains settings made on the championship setup page and the task setup page and also maintains a list of pilot names / competition numbers / device ID’s that it has seen to be associated with the championship.

 

The local device configuration file 

This is a file which is written to the device from the device setup page.  There can only be one of these files on the device.  The file is named logger.xml and it is placed in the root of the device drive.

 

The purpose of this file is to contain all the information necessary to download data from the device into the correct format.  It is placed on the device rather than on the host because it is then inherently portable between hosts.  The idea is that once a generated .igc file for a task has been produced on the organizer’s system, the same thing can be done on the pilot or team leader’s system. 

 

This file therefore contains some information from the master device configuration file and the championship configuration file.  This includes task window settings, so is modified whenever it is read by the organizer’s system. 

 

The file contains a hash of the organizer’s password (as set on the championship setup page) so a failed match (eg when it is being read by the team leader or pilot) flags this file as read-only.

 

When a device is connected to the host, the first thing it does is read the logger.xml file to see how the data on the device should be handled.  If there is no logger.xml file found, then the program goes to the device setup page so one can be created according to the specification in the championship configuration file and the master device configuration file for that particular type of device.

 

The device setup file 

This is a file which many of these devices require.  They are specific to the hardware and are usually a simple text file which is placed in the root of the drive which instructs the device which logging frequency to use, which NMEA sentences to save Etc.    The name, location and  ‘approved’ settings for this file are contained in the master device configuration file and it is written to the device whenever a local device configuration file is written to the device.

 

The Geochron, for example, needs a file named GLOGCON.TXT and the default settings in it are:

Mode = 0

Log What = RMC;GGA;GSA

Time Between Logs = 00:10:00

Holdoff = 5

WAAS = 0

 

The local device configuration file should contain a hash of the device setup file which should be checked whenever a device is connected to the host.  If the match fails, the operator should always be alerted that the device setup file has been altered because the data to be downloaded might not be in the form which is expected from other settings in the local device configuration file.

 

Data download

It is proposed that in principle the first thing that should happen once a device is connected to the host is that the software downloads all the data from the device to the host into a subdirectory of the path to where all cached files are saved as set in Settings.  This immediately establishes a complete backup of all data on the device.   The subdirectory should be named PILOT_name_x where x is the flight recorder number, (ie 1 for primary, 2 for secondary Etc. as stated in the local device configuration file.)

 

There are however several problems associated with this.  If it is a simple backup, the host will be flooded with vast quantities of data and it will be slow.  Better then, for a comparison to be made between the data already saved on the host and the data on the device, and just copy across anything new.

 

The problem with this is that some devices will make duplicate file names, and whatever happens, the overwriting or deleting of backed up data on the host must be avoided.  Typically, these devices will generate track files like LOG0.TXT, LOG1.TXT, LOG2.TXT Etc so it is not difficult to imagine a situation where duplicate file names will appear when the track files on a device have been cleared between tasks. 

 

It is therefore proposed that whatever the directory structure on the device, first the file names and creation dates of all files on the device are collected in a single flat list, and ordered by creation date.  These are then compared to files on the host which are named creationDate_originalFileName  thus LOG0.TXT which was created on 21 Jan 2009 at 12:34:51 becomes 20090121123451_LOG0.TXT

 

Any new files on the device are written to the backup cache (in a single directory, sub-directory structure on the device is ignored), files already copied are ignored, and there is no chance of duplicates.

 

(Note that although something needs to be done to address this problem, this particular scheme is a slightly speculative proposal because there is some evidence that the devices aren’t very reliable at writing accurate file creation dates.  This will become evident once some devices have been tested.)

 

It is considered possible, if not probable, that data corruption can occur if the connection between the device and host is broken whilst data is being transferred.  (ie the SD Card is removed from its slot, USB plug is pulled).  It is therefore proposed that the only time data actually passes across this boundary is during this backup process.  Reading the data required for the eventual production of the generated .igc file is done from the backup, not the device.

 

Data conversion

Once the data has been downloaded from the device, it must be converted to the format required in the CIMA specification and saved to a file with the correct name in the directory where the final .igc data is saved as set in Settings.

 

All the data from the source files needs to be collected together in one place and in chronological order, and then filtered according to the task window setting. 

 

The software should use an integrated version of GPS Babel to effect the actual data conversion.  The GPS Babel configuration settings for this type of device are included in the master device configuration file but are placed in the local device configuration file when the device is configured in Device Setup.  The local device configuration file settings are the ones used to convert the data.

 

Once the file is converted the final .igc file may need to have the correct headers attached to it as specified in the CIMA specification.

 

General use

It is expected the ‘home page’ of the application is the Data download page so it is ready to start work immediately the application is started.  If some problem is found, for example there is no currently configured championship, then the application will automatically switch to that page. 

 

It must be remembered that the operators of this software will typically be very inexperienced in what is actually a task fundamental to the success of the championship, and they will be under some pressure to complete the job with many distractions going on around them.  The software should therefore be designed to continuously inform the operator what is going on, and must be recoverable from as many errors as possible (why the backup of the entire contents of the device is so important).

Summary.

This looks awfully complicated but it is hoped the principle at least is relatively simple.  An attempt has been made here to specify a system which in its simplest form should not be outlandishly complicated to create but is intrinsically extensible once it has been created.

Annex 1

Some notes about existing data download software.

 

The purpose is to collect the track (and gps altitude) a pilot flew in a task into a file with a standard format and name which can be easily read by an analysis program in a consistent way.

 

The file name is an important part of the specification as it is used by flight analysis programs to identify who the track belongs to, in what task, and whether it is their primary or backup track recording.

 

They are set up so at initial registration the pilot name and competition number are matched with the serial numbers of his loggers.  Whenever the logger is subsequently connected to the software it can then identify who it belongs to and the correct file name can be generated.

 

The task number is also part of the required file name.  Existing software are set up so the operator sets the task number of the current batch of loggers to download and this is used when the file name is constructed.

 

Whenever a logger is connected to a computer, -all- the data on it is saved to the host computer.  This ‘hidden data’ has rescued more than one pilot from getting a ‘null’ score after his correctly generated track mysteriously got deleted. 

 

Often the raw data on the logger will not just contain track data for the task in question, but earlier flights, perhaps days or weeks before.  It also may be that the logger will start overwriting earlier track fixes once the memory is full so all the fixes associated with a particular flight may not be in a contiguous block or in the order they were flown.  Either way, this will cause difficulties in flight analysis, so track fixes are always written to the generated file in correct chronological order, and there is a ‘filter’ feature which is set up by the operator so only track items included within a specific time span are written to the generated file.  This time span is usually set to something like 30 minutes before the task started to 30 minutes after it ended.  This usually guarantees that only the track the pilot flew in the task being included in the track data to be analysed.

 

The organization NEVER deletes data from a logger.  This is for several reasons, but mainly because it is not very unusual for loggers to somehow get from the ‘unread’ box to the ‘read’ box without being read.  If the data has not been deleted then it can always be downloaded again later.  It is also fairly common that the pilot or team leader will want to download the data for their own use after the logger has been released by the organization.  The down-side of this is that it rather common that tracks are never deleted from a flight recorder so ‘wrapped’ tracks in loggers like the MLR with a limited amount of memory are quite common, and is why the task ‘filter’ is so important.

 

For simplicity, normally, all times in track data are retained in UTC time.  We did however encounter a problem at WPC 2007 in China where tracks sometimes went through midnight UTC.  The IGC file format does not have a mechanism for a date change in the middle of a track and it caused a problem with the analysis.  We found the easiest solution in this case was to time-shift all track fixes to local time.