Data Documentation
Welcome to the data documentation portal . In these pages you will find information about what the numbers provided in the GFO data wesbites mean.
Data Products
In this section we briefly detail the various data products in the fireball data reductionprocess, from the raw images captured by the observatories to very high-level information like orbit and trajectory characteristics. The difference between the two extremes is ~10 orders of magnitude!
Data Products summary table
| Data Product level | Description | Retention period | Typical disk space used (Bytes GFO-1 year-1) |
|---|---|---|---|
| L0 | Raw images | months, years if ingested onto HPC data centre | 1015 |
| L1A | Detected events + calibration frames | years | 1013 |
| L1B | Engineering logs | duration of the project | 1011 |
| L2 | Raw images for corroborated events | indefinite | 1012 |
| L3 | Astrometry + Photometry | indefinite | 107 |
| L4 | Triangulation, orbit, darkflight… | indefinite | 107 |
| L5 | Key Parameters | indefinite | 106 |
Key parameters - L5 data product
The L5 data product (key parameters) are high level information about the fireball/meteoroid, such as its orbital parameters, what its duration was, what height it started…
Show Table
| Column Name | notation | Format | Unit | UCD | Comment |
|---|---|---|---|---|---|
| General information | |||||
| event_codename | DNYYMMDD_01 | Unique meteor identifier used in the DFN pipeline | |||
| event_nickname | free string | Nickname given to meteor event | |||
| num_cams_geometry | Number of cameras used to triangulate (pure geometry) | ||||
| num_cams_timing | Number of observation viewpoints (cameras) with timing | ||||
| processing_triangulate_version | Version of the trajectory software used | ||||
| Atmospheric entry | |||||
| datetime | ISO8601 | time.epoch | Time expressed in UTC timescale at the first observed bright flight point | ||
| JD | day | time.start | Julian Day at the first observed bright flight point | ||
| slope | deg | Angle between the trajectory and horizontal plane | |||
| duration | s | time.duration | duration of the visible bright flight | ||
| fragmented | Fragmentation visible during bright flight? (Y/N) | ||||
| Initial | |||||
| initial_velocity $\pm$ err_initial_velocity | $v_{i} \pm v_{i}$ | m/s | Speed at first observed bright flight point | ||
| initial_height | $H_{i}$ | m | Height of the first observed bright flight point | ||
| initial_longitude | $\lambda_{i}$ | decimal (WGS84) | deg | Approximate longitude of the first observed bright flight point | |
| initial_latitude | $\phi_{i}$ | decimal (WGS84) | deg | Approximate latitude of the first observed bright flight point | |
| initial_slope $\pm$ initial_slope_err | deg | Angle between the trajectory and horizontal plane at the last observed bright flight point | |||
| initial_bearing $\pm$ initial_bearing_err | deg | Direction the meteor is heading (East of North). Should be radiant+180 degrees. | |||
| Final | |||||
| final_velocity $\pm$ err_final_velocity | $v_{e}\pm v_{e}$ | m/s | Speed at last observed bright flight point | ||
| final_height | $H_{e}$ | m | Height at the last observed bright flight point | ||
| longitude_end | $\lambda_{e}$ | decimal (WGS84) | deg | Approximate longitude of the last observed bright flight point | |
| latitude_end | $\phi_{e}$ | decimal (WGS84) | deg | Approximate latitude of the last observed bright flight point | |
| final_slope $\pm$ final_slope_err | deg | Angle between the trajectory and horizontal plane at the last observed bright flight point | |||
| final_bearing $\pm$ final_bearing_err | deg | Direction the meteor is heading (East of North) | |||
| H_peak_brightness | $H_{max}$ | m | Height of peak brightness | ||
| Orbit | |||||
| orbit_type | heliocentric, geocentric, or hyperbolic… unstable = multiple monte carlo clones gives different results | ||||
| semi_major_axis $\pm$ err_semi_major_axis | $a$ | AU | phys.size.smajAxis | ||
| eccentricity $\pm$ err_eccentricity | $e$ | src.orbital.eccentricity | |||
| inclination $\pm$ err_inclination | $i$ | deg | src.orbital.inclination | ||
| argument_periapsis $\pm$ err_argument_periapsis | $\omega$ | deg | |||
| longitude_ascending_node $\pm$ err_longitude_ascending_node | $\Omega$ | deg | |||
| true_anomaly $\pm$ err_true_anomaly | deg | ||||
| perihelion $\pm$ err_perihelion | $q$ | AU | |||
| aphelion $\pm$ err_aphelion | $Q$ | AU | |||
| longitude_perihelion $\pm$ err_longitude_perihelion | $\varpi$ | deg | |||
| T_j $\pm$ err_T_j | $T_j$ | Tisserand criterion wrt Jupiter | |||
| Radiant | |||||
| sol_longitude | $\lambda _0$ | decimal | deg | Solar longitude at time of entry | |
| RA_inf | $\alpha _{inf}$ | decimal | deg | pos.eq.ra | Right Ascension of the apparent radiant |
| Dec_inf | $\delta _{inf}$ | decimal | deg | pos.eq.dec | Declination of the apparent radiant |
| RA_g $\pm$ err_RA_g | $\alpha _g$ | decimal | deg | pos.eq.ra | Right Ascension of the Geocentric radiant |
| Dec_g $\pm$ err_Dec_g | $\delta _g$ | decimal | deg | pos.eq.dec | Declination of the Geocentric radiant |
| V_g $\pm$ err_V_g | $v _g$ | m/s | Velocity at 1 AU (assuming Earth not there) | ||
| meteor_shower | $IAU #$ | associated IAU meteor shower code (3 capsized letters, eg. NTA) |
Strewn field
The fall positions of meteorites on the ground is affected by a number of factors. Although there can be some exceptions, meteorite dropping meteoroid stop being visible between 15 and 35 km altitude, the typical height they have slowed down below the ablation limit (~3 km/s). This means that in order to derive fall positions we must numerically predict the meteoroid’s position, based on how it is going to be affected by the winds. Even if a meteoroid is tracked to a precision of tens of metres during the brightflight phase, its position uncertainty on the ground can grow to several hundreds of metres.
Astrometry and photometry - L3 data product
note: The preferred format for sharing meteor/fireball astrometry and photometry is the GFE standard developed by Jim Rowe and colleagues.
Below is documentation for the DFN legacy astrometric ECSV format.
L3 data products are time series of meteor observations, bundled per observation viewpoint (camera). The meteor/fireball is observed multiple times throughout its flight, leading to a series astrometric and photometric measurements.
Astrometry: which direction in space was it? (there are different possible reference frames and coordinates systems to express this, but they come down to two angles representing the direction).
Photometry: how bright was it? (here again different ways to express this, although a common one is using stellar magnitudes).
Keywords in bold constitude the minimum required dataset for being able to use a datafile in the DFN data pipeline (outlined here for interoperability with other sources of data). If you need to manually generate an astrometric observation file for use in the DFN pipeline, you can use this python jupyter notebook.
Metadata
The metadata for an observation table consist of a set of key-value pairs, describing information about the environment the image was taken in, and what calibration methods were later applied to the date.
Show Table
| Key name | Format | Unit | UCD | Comment | |
|---|---|---|---|---|---|
| obs_longitude | decimal | deg | pos.earth.lon | Longitude of the observer | |
| obs_latitude | decimal | deg | pos.earth.lat | Latitude of the observer | |
| obs_elevation | m | pos.earth.altitude | Elevation above sea level of the observer | ||
| camera_codename | Camera codename. eg. DFNSMALL43 | ||||
| location | Camera location. eg. Forrest | ||||
| event_codename | Event codename. Unique identifier for events across the GFO. Format: DNYYMMDD_seq. eg. DN150417_01 | ||||
| image_name | obs.image | File used for point picking. eg. 43_2015-04-17_200359_DSC_1270-G.fits | |||
| NAXIS1 | Number of pixels on the x-axis | ||||
| NAXIS2 | Number of pixels on the y-axis | ||||
| camera_filter | Color array original pattern | ||||
| cropped | Image is a crop | ||||
| exposure_time | s | Exposure time | |||
| aperture | Lens aperture setting | ||||
| focal | m | Lens focal length | |||
| iso | Camera ISO setting | ||||
| instrument | Camera type | ||||
| lens | Lens model | ||||
| lens_type | Lens type: rectilinear, fish-eye… | ||||
| jd_start_obs | day | Timestamp of shutter opening (Julian Day) | |||
| isodate_start_obs | ISO8601 | Timestamp of shutter opening in ISOT format scaled in UTC | |||
| tracking | bool | Sidereal tracking enabled? | |||
| observer | |||||
| origin | Corresponds to the ORIGIN FITS header keyword | ||||
| telescope | Name of the imaging system | ||||
| processing_filter | Pre-processing method used for de-Bayering | ||||
| color_channel | em.opt | Color channel used. eg. G, B, R $\neq$ standard B, I, R, U, V | |||
| point_picking_user | Username of the person who point picked | ||||
| point_picking_hostname | Computer used for point picking | ||||
| point_picking_comment | User comment for the point picking | ||||
| point_picking_write_time | time.processing | UTC Timespamp of last point picking | |||
| shutter_inversion | bool | Shutter open when should be close and vice versa? (True / False) | |||
| astrometry_number_stars | Number of stars used for calibration | ||||
| astrometry_calibration_image | Image used for calibration | ||||
| astrometry_catalog_efficiency | Percentage of non-spurious stars in the catalog | ||||
| astrometry_global_correction | Fine astrometric corrections applied | ||||
| astrometry_catalog | meta.software | Catalog used for astrometry (actual catalog or image) | |||
| astrometry_method | meta.software | Calibration method (local, global…) | . | ||
| photometry_raw_software | meta.software | Software used to compute raw photometry | |||
| photometry_raw_method | meta.software | Meteor photometry method | |||
| photometry_raw_write_time | time.processing | UTC Timespamp of photometry calculation | |||
| photometric_zero_point | photometric zero-point | ||||
| photometric_zero_point_uncertainty | $1-\sigma$ uncertainty of the zero-point | ||||
| photometric_zero_point_datetime | Date of the image in which photometric zero-point was calculated | ||||
| astrometry_write_time | ISO8601 | time.processing | UTC Timespamp of conversion from pixels to sky | ||
| triangulation_method | meta.software | Triangulation method | |||
| triangulation_write_time | ISO8601 | time.processing | UTC Timespamp of triangulation runtime | ||
| triangulation_file_N | observation files used for triangulation $N \in {2,\infty}$ | ||||
| trajectory_analysis_velocity_order | Order of computation of velocities (1 = consecutive observations, 2 = starts together and ends together, $> 2$ = undefined) | ||||
| triangulation_ra_ecef_inf | decimal | deg | RA of radiant at infinity in ECEF frame | ||
| triangulation_ra_ecef_inf_err | decimal | deg | error in RA of radiant at infinity in ECEF frame | ||
| triangulation_dec_ecef_inf | decimal | deg | Dec of radiant at infinity in ECEF frame | ||
| triangulation_dec_ecef_inf_err | decimal | deg | error in Dec of radiant at infinity in ECEF frame | ||
| triangulation_ra_eci_inf | decimal | deg | RA of radiant at infinity in ECI frame | ||
| triangulation_ra_eci_inf_err | decimal | deg | error in RA of radiant at infinity in ECI frame | ||
| triangulation_dec_eci_inf | decimal | deg | Dec of radiant at infinity in ECI frame | ||
| triangulation_dec_eci_inf_err | decimal | deg | error in Dec of radiant at infinity in ECI frame | ||
| triangulation_slope_inf | decimal | deg | Slope of the radiant at infinity in ENU frame | ||
| triangulation_slope_inf_err | decimal | deg | error in slope of the radiant at infinity in ENU frame | ||
| triangulation_azimuth_inf | decimal | deg | Azimuth of the radiant at infinity in ENU frame | ||
| triangulation_azimuth_inf_err | decimal | deg | error in Azimuth of the radiant at infinity in ENU frame | ||
| ballistic_alpha | dimensionless ballistic parameter $\alpha$ | ||||
| ballistic_beta | dimensionless mass loss parameter $\beta$ | ||||
| ballistic_entry_mass | kg | entry mass calculated using MGs ballistic parameters given initially spherical body of density 3500kg/m3 | |||
| ballistic_alpha_all | dimensionless ballistic parameter $\alpha$ calculated using all cameras | ||||
| ballistic_beta_all | dimensionless mass loss parameter $\beta$ calculated using all cameras | ||||
| ballistic_entry_mass_all | kg | entry mass calculated using MGs ballistic parameters from all cameras given initially spherical body of density 3500kg/m3 | |||
| ballistic_reference_velocity | $\mbox{m s}^{-1}$ | v0 used in Gritsevich fitting to normalise velocities | |||
| EKS_smoother_run | bool | True/False | |||
| EKS_initial_velocity_all_cam | $\mbox{m s}^{-1}$ | Entry velocity calculated using multiple cameras. Not part of pipeline. May be run on chosen cameras. | |||
| EKS_initial_velocity_all_cam_err | $\mbox{m s}^{-1}$ | 1 sigma error on entry velocity calculated using multiple cameras within EKS. |
Columns
Show Table
| Column Name | Format | Unit | UCD | Comment |
|---|---|---|---|---|
| __Point picking__ | ||||
| datetime | ISO8601 | time.epoch | Time expressed in UTC timescale | |
| JD | day | time.epoch | Julian Day | |
| time_err_plus | sec | |||
| time_err_minus | sec | |||
| x_image | pix | Physical X coordinate. Origin: left | ||
| y_image | pix | Physical Y coordinate. Origin: top | ||
| err_plus_x_image | pix | |||
| err_plus_y_image | pix | |||
| err_minus_x_image | pix | |||
| err_minus_y_image | pix | |||
| x_fits | pix | Physical X coordinate in the FITS standard (Y inversed) | ||
| y_fits | pix | Physical Y coordinate in the FITS standard (Y inversed) | ||
| err_plus_x_fits | pix | |||
| err_plus_y_fits | pix | |||
| err_minus_x_fits | pix | |||
| err_minus_y_fits | pix | |||
| de_bruijn_sequence_element_index | index in the De Bruijn sequence | |||
| dash_start_end | ‘start’ or ‘end’ | |||
| periodic_pulse | periodic pulse is for future PFSK data | |||
| pick_flag | ‘M’: for manually picked | |||
| encoding_type | ‘PW’: pulse width, ‘PF’: pulse frequency | |||
| __Brightness__ | ||||
| brightness_dash_V | ADU | Integrated brightness over dash in V band | ||
| err_plus_brightness_dash_V | ADU | |||
| err_minus_brightness_dash_V | ADU | |||
| flux_dash_V | ADU | Integrated brightness over dash in V band, normalised to exposure time | ||
| err_plus_flux_dash_V | ADU | |||
| err_minus_flux_dash_V | ADU | |||
| m_V | mag | apparent magnitude in V band | ||
| err_plus_m_V | mag | |||
| err_minus_m_V | mag | |||
| M_V | mag | absolute magnitude in V band (zero point @100km altitude) | ||
| err_plus_M_V | mag | |||
| err_minus_M_V | mag | |||
| __Calibration__ | ||||
| azimuth | decimal | deg | pos.az.azi | East of North azimuth |
| err_plus_azimuth | decimal | deg | pos.az.azi | |
| err_minus_azimuth | decimal | deg | pos.az.azi | |
| altitude | decimal | deg | pos.az.alt | Angle above horizon (geometric, not apparent) |
| err_plus_altitude | decimal | deg | pos.az.alt | |
| err_minus_altitude | decimal | deg | pos.az.alt | |
| zenith_angle | decimal | deg | pos.az.zd | Angle from zenith |
| err_plus_zenith_angle | decimal | deg | pos.az.zd | |
| err_minus_zenith_angle | decimal | deg | pos.az.zd |
Misc
Event codenames
Events are automatically assigned a codename, based on the UTC date they happened at, following the format DNYYMMDD_SS:
- DN is a historical prefix for “Desert Network”. However it applies to all Global Fireball Observatory events regardless.
- item YY corresponds to the last 2 digits of the year
- item MM corresponds to the month
- item DD corresponds to the day
- item SS is a sequence number. It does not follow any particular order (not even chronologically). The first event ingested for that day gets 01 as sequence number, other events are incremented from that.
Event folder structure
This is an example of a typical GFO event file strucure:
DN181225_03
├── 033_Ait_Ben_Haddou | system-number_camera-location
│ ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment_altaz.png | plot of alt/az coordinates in
celestial sphere (from astrometric file)
│ ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment.ecsv | astrometric file
│ ├── 033_2018-12-25_183159_E_DSC_0064-G.fits | fits conversion of the fireball image
│ ├── 033_2018-12-25_183159_E_DSC_0064.NEF | raw fireball image
│ ├── 033_2018-12-25_183159_E_DSC_0064.thumb.2400.400.tile.jpg | detection tile from the camera (usually 400x400 greyscale image)
│ ├── 033_2018-12-25_183159_E_DSC_0064.thumb.jpg | jpeg colour extract of the fireball image
(warning: it is not exactly the same size the the RAW or FITS image)
│ ├── 2018-12-25_DFNEXT033_log_interval.txt | camera capture control log file
│ ├── dfnstation.cfg | camera station file
│ ├── calib | folder containing calibration data
│ │ ├── 033_2018-12-25_193029_E_DSC_0181-G_fh.fits | calibrated image (FITS header contains calibration coefficients)
│ │ ├── 033_2018-12-25_193029_E_DSC_0181-G_fh.matched_stars | list of matched stars
│ │ ├── 033_2018-12-25_193029_E_DSC_0181.NEF | raw calibration image
│ │ └── dfnstation.cfg | camera station file
│ └
│
├── eventflags.cfg | config file for all manual entries (event nickname, search status...)
│
├── WRF/
│ ├── profile_DN151127_01_1045.csv | 1D atmospheric profile centered on last astrometric point
│ ├── profile_DN151127_01_1045.pdf | plot of the above file
│ └── wrfout_d04_2015-11-27_06:00:00 | full 3D WRF atmospheric model
│
├── trajectory_auto_20190108 | automatic trajectory folder generated
│ ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment.ecsv | astrometric file
│ ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment_point_picking_plot.jpg
| coulour plot of centroids on the image
│ ├── 2019-01-08_DFN_WS_log_orbit.txt | log file for the orbit calculation
│ ├── 2019-01-08_DFN_WS_log_trajectory.txt | log file for the trajectory determination and brightflight analysis
│ ├── 45_2018-12-25_183158_S_DSC_0058-G_DN181225_03_2019-01-07_111647_hadry_nocomment.ecsv
│ ├── 45_2018-12-25_183158_S_DSC_0058-G_DN181225_03_2019-01-07_111647_hadry_nocomment_point_picking_plot.jpg
│ ├── DN181225_03_key_parameters.yaml | key parameters for the event
│ ├── DN181225_03_MC_orbits.csv | Monte Carlo orbits calculated
│ ├── DN181225_03_MC_orbits.pdf | ecliptic orbital plot
│ └── DN181225_03_trajectory.kmz | Google Earth KMZ trajectory file
└──