FRMFILE
Home ›
Software ›
Manual Pages ›
FRMFILE
Section: File Formats (5)
Updated:
Index
Return to Main Contents
NAME
frmfile - frame file format
SYNOPSIS
#include "/usr/neuro/src/run.h"
DESCRIPTION
The frame file is the key file in the set of data,
collectively known as a
runfile,
produced or used by
dsepr(1),
analysis(1)
and related programs.
It is the one that holds information on both triggered and
untriggered channels, as well as the data
collected from the triggered channels.
Frame File Format
+--------------------------------------+
| |
| Run Header |
| |
+--------------------------------------+
+--------------------------------------+
| Frame Header 1 |
+--------------------------------------+
| Frame Data |
| |
+--------------------------------------+
...
+--------------------------------------+
| Frame Header n |
+--------------------------------------+
| Frame Data |
| |
+--------------------------------------+
Files from a Captured Run
Each run of data results in one frame file, and a waveform file
for each untriggered channel.
The frame file will always begin
with a run header, which records general information about the run,
such as sampling rate,
number of frames, and for each channel, the sampling rate divisor,
and calibration information.
Captured frames of triggered channel
data follow.
Each frame begins with a frame header which indicates
the starting sample number (i.e. at what time within the run the
trigger pulse occurred).
The frame also contains one trace (a
series of A/D samples) for each triggered channel.
Each waveform
file contains a contiguous series of A/D samples for one untriggered
channel.
Frame File for Averaged Data
Averaged data, produced by either real-time or post-hoc averaging,
are stored in a frame file of the same format as that used for raw
data.
A value in the run header indicates the method used to obtain
the average.
One frame is recorded for each bin in a multi-bin
average.
Instead of recording the starting sample number, the
frame header records the number of sweeps of raw data that were
averaged in that bin.
Frame Format
+-------------------+-------------------+
| Flags | Tag |
+-------------------+-------------------+ Frame Header
| Sample no. or No. of frames |
+---------------------------------------+
+---------------------------------------+
| Frame Data |
| for trace 0 |
+---------------------------------------+
...
+---------------------------------------+
| |
| Frame Data |
| for trace n-1 |
| |
+---------------------------------------+
Frame Header
The frame header has three fields:
deletion flags, a tag code,
and a number.
This number is either a sample number indicating
when the raw frame was triggered, or a count of the number of sweeps
averaged into the frame.
Frame Data
The sample points for one triggered sweep from each triggered
channel are stored after the frame header.
Each sweep is stored
contiguously.
Since channels can have independent sampling rate
divisors, the sweeps in a frame may not all contain the same number
of data points, even though they all represent the same length of
time.
However, the number of points per sweep for any given
triggered channel is consistent throughout all frames.
Binary Data Formats
A/D data samples in the frame file and waveform files are stored
as 16 bit signed integers, 2's complement for negatives, and most
significant byte first (known as big-endian).
Numbers in the run
header and frame headers are either long integers (32-bit, 2's
complement signed, big-endian), short integers (16-bit, 2's
complement signed, big-endian), double precision floating-point
(64-bit, IEEE format, big-endian), or character (fixed length
arrays, single-byte ASCII, NUL terminated).
Below are the C Language structure definitions used for the headers.
They are appropriate for big-endian architectures (Motorola, SPARC, PowerPC).
For little-endian architectures (DEC, Intel) it is necessary to flip
the byte ordering for all short ints, long ints and doubles.
Listing 1: run.h
____________________________________________________________
/*
* run.h -- definitions of run file headers and related info.
*
* (c) 1988, G.R. Detillieux, Spinal Cord Research Centre,
* The University of Manitoba. All Rights Reserved.
*/
#define NCHFRM 16 /* # chans for frame (triggered) data */
#define NCHREG 16 /* # chans for regular (untriggered) data */
#define CHNAMESIZ 42 /* size of channel name string */
typedef short int ADSAMPLE; /* data type used for A/D values */
typedef long int ADPERIOD; /* data type used for time values (sampling periods) */
/* calibration info for one channel: */
struct calinfo {
ADSAMPLE ca_zero; /* A/D value for zero volts */
ADSAMPLE ca_height; /* A/D value for cal pulse minus value for zero */
long int ca_level; /* level of cal pulse in uV */
short int ca_gain; /* gain factor used for channel */
char ca_name[CHNAMESIZ]; /* channel name string */
};
#define RUNMAGIC 0xFFAAFABF /* "magic" number identifying run file */
/* define averaging methods used by dsepr, others in analysis/param.h: */
#define AM_TRAWCAP 0 /* raw trace capture */
#define AM_TAVGCAP 1 /* averaged trace capture, ... */
/* ... same code as AM_TAVGFRMLST */
/* run header in frame data file: */
struct runhdr {
long int rh_magic; /* magic number for run file */
ADPERIOD rh_length; /* run length as #samples */
double rh_samprate; /* sample rate in Hz */
long int rh_nframes; /* # frames in file */
long int rh_frmsiz; /* frame size in bytes */
ADPERIOD rh_delay; /* delay to start of frame as #samples */
ADPERIOD rh_window; /* frame sampling window as #samples */
ADPERIOD rh_gpper; /* gate pulse period as #samples */
ADSAMPLE rh_minbinlevel; /* min. W.F. level for AM_TAVGLEVEL */
ADSAMPLE rh_maxbinlevel; /* max. W.F. level for AM_TAVGLEVEL */
short int rh_avgmethod; /* averaging method, or 0 for raw */
short int rh_levelwf; /* W.F. from which above levels come */
ADPERIOD rh_wreduce; /* #samples by which usable window reduced */
long int rh_starttime[2];/* time capture started, if available */
short int rh_reserve[19]; /* reserved for future use */
short int rh_needrhdfile; /* extra run header info. needed */
short int rh_npts[NCHFRM]; /* # points in triggered channel frames */
short int rh_frmdiv[NCHFRM]; /* sample rate divisors for trig. chan. */
short int rh_regdiv[NCHREG]; /* sample rate divisors for untrig. chan. */
short int rh_frmchan[NCHFRM]; /* channel numbers for traces */
short int rh_regchan[NCHREG]; /* channel numbers for waveforms */
struct calinfo rh_frmcal[NCHFRM]; /* calibr. info for framed channels */
struct calinfo rh_regcal[NCHREG]; /* calibr. info for regular (unframed) channels */
long int rh_frmres[NCHFRM]; /* reserved for future use */
long int rh_regres[NCHREG]; /* reserved for future use */
};
/* header at beginning of each frame: */
struct frmhdr {
long int fh_flags; /* tag, deletion flags & room for more */
ADPERIOD fh_sampnum; /* sample # where gate pulse detected */
};
#define fh_numsweeps fh_sampnum /* # of sweeps in an averaged frame */
/* frame header flags: */
#define DELFLAGS 0xE0000000 /* all 3 deletion flags */
#define MANDEL 0x80000000 /* frame manually deleted */
#define AUTODEL1 0x40000000 /* frame auto-del'd due to clipping */
#define AUTODEL2 0x20000000 /* frame auto-del'd due to bad cal pulse */
#define ISDELETED(fh) ((fh).fh_flags & DELFLAGS)
#define TAGMASK MAXSHORT /* all bits reserved for tags */
#define NTAGS 4096 /* # of different tags used */
#define INTAGLIST(fh,tlist,tlen) (inlist((unsigned short int)((fh).fh_flags & TAGMASK), \
(tlist), (tlen)))
____________________________________________________________
Calibration and unit conversions
Note that the fields rh_frmcal and rh_regcal are both arrays of
structures containing the calibration information and identifying
name for each channel.
The rh_frmcal is used for the framed, or
triggered channels, whose samples are contained in the frames of
this same file, right after the run header.
The rh_regcal is used
for the regular, or untriggered channels, whose samples are contained
in the separate waveform files.
These structures are copied from the calibration file at the start of
the data capture.
The calibration file (default.cal) simply contains an array of 16 or more of
these structures, one for each channel on the A/D converter,
with no header or other fields in the file.
The structures for the channels to be captured are copied to the
rh_frmcal and rh_regcal entries corresponding to the traces or waveforms
to which these channels are assigned.
Each of these structures contains three fields that permit conversion
from A/D sample values to voltages.
The ca_zero represents a zero
volt level on the channel, expressed as an A/D sample value.
The
ca_height represents the height of a calibration pulse of known
amplitude, expressed as a displacement in A/D sample values.
The
ca_level is the amplitude of this same calibration pulse in
microvolts.
Thus, to convert A/D samples for a given untriggered
channel "i" to millivolts, one could use this formula:
(samp - (ca_zero of rh_regcal )) × (ca_level of rh_regcal )
i i
mv = -----------------------------------------------------------
(ca_height of rh_regcal ) × 1000
i
the equivalent in C Language, with required type conversions, would be:
-
mv = (samp-run.rh_regcal[i].ca_zero)
* (double)run.rh_regcal[i].ca_level
/ ((double)run.rh_regcal[i].ca_height * 1000.0);
Two other fields in the calibration information structures are
ca_gain and ca_name.
The ca_gain is simply the gain code used for
the A/D channel used for this signal.
It is provided for information
only, and is not required for level conversions of the A/D samples.
The ca_name is an ASCII NUL terminated character string containing
the name given to the channel - usually describing the signal.
Similar unit conversions must be done on the time axis, to convert
from sample numbers to, say, milliseconds.
The rh_samprate contains
the base sampling rate at which the run was captured, so to calculate
the the time of onset of the n-th sample at this rate, in milliseconds,
the C formula would be:
-
ms = n * 1000.0 / run.rh_samprate; [1]
Complicating matters somewhat is the fact that samples are not
necessarily stored at the base sampling rate for all channels.
In
fact, the channels may all be stored using different effective
rates.
The rh_frmdiv and rh_regdiv arrays contain the sampling
rate divisors for all triggered and untriggered channels, respectively.
These divisors are all integers, and determine how samples are
stored for the channels.
A divisor of 1 indicates that all samples
are stored, at the base sampling rate.
A divisor of n indicates
that only the first of every n samples at the base rate is kept
for the channel.
As a special case, a 0 divisor means this particular
channel was not used.
The complete C formula to calculate the time of onset of the n-th
stored sample for untriggered channel "i" is:
-
ms = n * 1000.0 / (run.rh_samprate
/ run.rh_regdiv[i]); [2]
The sampling rate division of triggered channels is handled similarly,
except that the whole process is restarted for each frame.
Trigger
pulses on the trigger signal (which is not usually stored in the
run) cause frames to be started, triggering sweeps from each of
the triggered channels.
When a trigger pulse occurs during the
capture of a raw run file, the time of onset of the trigger is
recorded in the fh_sampnum field of the new frame's header, as a
sample number at the base sampling rate.
Thus, the first time
conversion formula above [1] can be used to get the time of onset
in milliseconds.
From the time of onset of the trigger pulse, the capture program
waits for a length of time specified by the rh_delay field, then
begins storing samples from the triggered channels into the new
frame, for the length of time specified by the rh_window field.
Both fields specify time as a number of samples at the base sampling
rate.
A negative delay indicates that pre-trigger sampling was
performed in the frames.
For any triggered channel with a divisor
n greater than 1, only the first of every n samples is kept, starting
with the first sample after the delay has passed.
Since triggered
channels can be stored at different rates, but for the same length
of time in each frame, the number of sample points per frame can
vary from channel to channel.
The rh_npts array indicates the
number of samples per frame for each triggered channel.
The time offset of the n-th sample of channel "i" from the start
of the frame can be calculated with a variant of formula [2], using
rh_frmdiv rather than rh_regdiv.
The time offset of the n-th sample
in a given frame, from the start of the run is:
-
ms = (frm.fh_sampnum + run.rh_delay
+ (n * run.rh_frmdiv[i]))
* 1000.0 / run.rh_samprate; [3]
Extended run header file format
For runs of data that have more than 16 waveforms, more than 16 traces,
or calibration zero or height values that are larger than 16 bits,
the run header format described above is not sufficient.
Rather than changing the format, which would break backwards compatibility,
we use this same header format,
in the
.frm
(frame) file,
for the first 16 traces and waveforms, but also add another
.rhd
run header file, which is an ASCII text file that contains run header
information on all traces and waveforms in use for the run, as well as other
parameters common for the entire run.
The programs that open a
runfile
will check for the presence of this run header file, and if there they
will also verify that the information in it is consistent with the run
header in the frame file, for all overlapping parameters.
The contents of this run header file consists of lines of parameter settings,
much like shell variables, in the form
-
NAME='value'
The names are all upper case variants of the names in the
runhdr
structure shown above, e.g.:
LENGTH,
SAMPRATE,
NFRAMES,
FRMSIZ,
DELAY,
WINDOW,
GPPER,
MINBINLEVEL,
MAXBINLEVEL,
AVGMETHOD,
LEVELWF,
WREDUCE
and
NEEDRHDFILE.
(This latter field is set to 1 if the run header file is needed.)
There are also variable names in the form
FRMDIV_n
and
REGDIV_n
(where
n
is the trace or waveform number, from 0 up to a maximum of 99),
for all trace or waveform specific run header parameters.
These include
NPTS,
FRMDIV,
FRMCHAN,
FRMCALZERO,
FRMCALHEIGHT,
FRMCALLEVEL,
FRMCALGAIN,
FRMCALNAME,
REGDIV,
REGCHAN,
REGCALZERO,
REGCALHEIGHT,
REGCALLEVEL,
REGCALGAIN
and
REGCALNAME.
Finally
RESERVED_n
is set for any of the reserved fields which are non-zero, for the sake
of preserving backwards compatibility for any future uses of these fields.
Start time field format
In 2015 we added a rh_starttime field to the run header, to track the actual
time the data capture started for a given run.
Before this, we could only estimate this based on the modification time of the
oldest waveform files for a run, as anything else was subject to subsequent
modification.
We use a standard Unix-style time_t format data type for this, which records
seconds since midnight UTC of January 1st, 1970.
The problem with this data type is it is in transition from a 32-bit integer
to a 64-bit one, because the 32-bit data type will overflow early in the year
2038.
Because not all systems currently use the 64-bit type (Mac OS X was one of the
first to make the transition), we define the field as an array of two 32-bit
long integers, and handle the machine-dependent conversion in software.
(See /usr/neuro/src/lib/runio.c for examples of this.)
When displaying this time, e.g. in dumprun, we use the C library's localtime()
function to convert to something human-readable.
By storing the time in UTC internally, we have a start time that is globally
accurate even when data files travel across time zones.
This is in contrast to the start date and time stored by Axoscope or pClamp
in the ABF file's header, which is stored in local time.
This can lead to some confusion if ABF files travel across time zones and are
then converted by axon2run, as the resulting run file's start time will not
accurately reflect the true local start time of the ABF file's capture.
It is best to convert ABF files in the time zone in which they were captured,
which can be overridden in software when running axon2run.
Raw direct-to-disk capture format
Programs like
dcap
and
dcavg,
or Concurrent's
dacq,
produce files in a raw direct-to-disk capture format.
These files can be used as input to
dsepr(1).
The format of the data in such a file is simply an array of A/D samples,
stored as 16 bit signed integers, 2's complement for negatives,
and in the machine's own native byte ordering.
The samples are stored in the order they are sampled from the A/D converter:
the first sample from the first channel, then the second channel, third
channel and so on, followed by the second sample from each of these channels,
and on.
There are no headers, and no separators or delimiters, just binary data.
Information about the number of channels, sampling frequency, and calibration
are not stored directly in these files, but must be kept track of separately.
This could be used as an intermediate format for a data conversion program,
which could then use
dsepr
to handle all the details of run file creation.
FILES
*.frm frame files, containing run headers
*.rhd run header file for extended file format
*.w?? corresponding waveform files
*.txt corresponding run descriptions
*.frd corresponding frame descriptions
SEE ALSO
dsepr(1), cap(1), analysis(1), dumprun(1), axon2run(1)
Index
- NAME
-
- SYNOPSIS
-
- DESCRIPTION
-
- Frame File Format
-
- Files from a Captured Run
-
- Frame File for Averaged Data
-
- Frame Format
-
- Frame Header
-
- Frame Data
-
- Binary Data Formats
-
- Listing 1: run.h
-
- Calibration and unit conversions
-
- Extended run header file format
-
- Start time field format
-
- Raw direct-to-disk capture format
-
- FILES
-
- SEE ALSO
-
This document was created by
man2html,
using the manual pages.
Time: 17:58:04 GMT, February 20, 2019
Copyright © G. R. Detillieux,
Spinal Cord Research Centre,
The University of Manitoba.