Data Documentation

Welcome to the data documentation portal . In these pages you will find information about what the numbers provided in the GFO data wesbites mean.

Data Products

In this section we briefly detail the various data products in the fireball data reductionprocess, from the raw images captured by the observatories to very high-level information like orbit and trajectory characteristics. The difference between the two extremes is ~10 orders of magnitude!

Data Products summary table
Data Product level Description Retention period Typical disk space used (Bytes GFO-1 year-1)
L0 Raw images months, years if ingested onto HPC data centre 1015
L1A Detected events + calibration frames years 1013
L1B Engineering logs duration of the project 1011
L2 Raw images for corroborated events indefinite 1012
L3 Astrometry + Photometry indefinite 107
L4 Triangulation, orbit, darkflight… indefinite 107
L5 Key Parameters indefinite 106

Key parameters

We call “key parameters” high level information about the fireball/meteoroid, such as its orbital parameters, what its duration was, what height it started…

Show Table
Column Name notation Format Unit UCD Comment
General information          
event_codename $#$        
num_cams_geometry         Number of cameras used to triangulate (pure geometry)
num_cams_timing         Number of observation viewpoints (cameras) with timing
num_cams_notstub         Number of observation viewpoints covering a significant proportion of the trajectory
Atmospheric entry          
datetime   ISO8601   time.epoch Time expressed in UTC timescale at the beginning of entry
JD     day time.start Julian Day at the beginning of entry
V_inf $v_{\infty}$   m/s   Velocity at the top of the atmosphere0
err_plus_V_inf $\pm v_{\infty}$   m/s    
err_minus_V_inf $\pm v_{\infty}$   m/s    
V_g $v _g$   m/s   Velocity at 1 AU (assuming Earth not there)
err_plus_V_g $$   m/s    
err_minus_V_g $$   m/s    
V_end $v_{e}$   m/s   Velocity at the end of bright flight (aka. observed terminal velocity)
err_plus_V_end $\pm v_{e}$   m/s    
err_minus_V_end $\pm v_{e}$   m/s    
H_inf $H_{\infty}$   m   Height at the start of entry
longitude_inf $\lambda_{\infty}$ decimal (WGS84) deg   Approximate longitude of the first bright flight point
latitude_inf $\phi_{\infty}$ decimal (WGS84) deg   Approximate latitude of the first bright flight point
H_end $H_{e}$   m   Height at the end of the bright flight (aka. observed terminal height)
longitude_end $\lambda_{e}$ decimal (WGS84) deg   Approximate longitude of the last bright flight point
latitude_end $\phi_{e}$ decimal (WGS84) deg   Approximate latitude of the last bright flight point
H_peak_brightness $H_{max}$   m   Height of peak brightness
slope     deg   Angle between the trajectory and horizontal plane
duration     s time.duration duration of the visible bright flight
fragmented         Fragmentation visible during bright flight? (Y/N)
M_V_max $MV_{max}$   mag   Maximum absolute magnitude of the fireball in V band
int_M_V $intMV$   mag   Integrated absolute magnitude of the fireball in V band
Orbit          
orbit_type         heliocentric, geocentric…
semi_major_axis $a$   AU phys.size.smajAxis  
eccentricity $e$     src.orbital.eccentricity  
inclination $i$   deg src.orbital.inclination  
argument_periapsis $\omega$   deg    
longitude_ascending_node $\Omega$   deg    
true_anomaly     deg    
perihelion $q$   AU    
aphelion $Q$   AU    
longitude_perihelion $\varpi$   deg    
T_j $T_j$       Tisserand criterion wrt Jupiter
Radiant          
sol_longitude $\lambda _0$ decimal deg   Solar longitude at time of entry
RA_inf $\alpha _{inf}$ decimal deg pos.eq.ra Right Ascension of the apparent radiant
Dec_inf $\delta _{inf}$ decimal deg pos.eq.dec Declination of the apparent radiant
RA_g $\alpha _g$ decimal deg pos.eq.ra Right Ascension of the Geocentric radiant
Dec_g $\delta _g$ decimal deg pos.eq.dec Declination of the Geocentric radiant
meteor_shower $IAU #$       associated IAU meteor shower code (3 capsized letters, eg. NTA)

Strewn field

The fall positions of meteorites on the ground is affected by a number of factors. Although there can be some exceptions, meteorite dropping meteoroid stop being visible between 15 and 35 km altitude, the typical height they have slowed down below the ablation limit (~3 km/s). This means that in order to derive fall positions we must numerically predict the meteoroid’s position, based on how it is going to be affected by the winds. Even if a meteoroid is tracked to a precision of tens of metres during the brightflight phase, its position uncertainty on the ground can grow to several hundreds of metres.

Astrometry and photometry

This section deals with observation files. Observations relate to a particular observatory. Keywords in bold constitude the minimum required dataset for being able to use a datafile in the DFN data pipeline (outlined here for interoperability with other sources of data). If you need to manually generate an astrometric observation file for use in the DFN pipeline, you can use this python jupyter notebook.

Metadata

The metadata for an observation table consist of a set of key-value pairs, describing information about the environment the image was taken in, and what calibration methods were later applied to the date.

Show Table
Key name Format Unit UCD Comment  
obs_longitude decimal deg pos.earth.lon Longitude of the observer  
obs_latitude decimal deg pos.earth.lat Latitude of the observer  
obs_elevation   m pos.earth.altitude Elevation above sea level of the observer  
camera_codename       Camera codename. eg. DFNSMALL43  
location       Camera location. eg. Forrest  
event_codename       Event codename. Unique identifier for events across the GFO. Format: DNYYMMDD_seq. eg. DN150417_01  
image_name     obs.image File used for point picking. eg. 43_2015-04-17_200359_DSC_1270-G.fits  
NAXIS1       Number of pixels on the x-axis  
NAXIS2       Number of pixels on the y-axis  
camera_filter       Color array original pattern  
cropped       Image is a crop  
exposure_time   s   Exposure time  
aperture       Lens aperture setting  
focal   m   Lens focal length  
iso       Camera ISO setting  
instrument       Camera type  
lens       Lens model  
lens_type       Lens type: rectilinear, fish-eye…  
jd_start_obs   day   Timestamp of shutter opening (Julian Day)  
isodate_start_obs ISO8601     Timestamp of shutter opening in ISOT format scaled in UTC  
tracking bool     Sidereal tracking enabled?  
observer          
origin       Corresponds to the ORIGIN FITS header keyword  
telescope       Name of the imaging system  
processing_filter       Pre-processing method used for de-Bayering  
color_channel     em.opt Color channel used. eg. G, B, R $\neq$ standard B, I, R, U, V  
point_picking_user       Username of the person who point picked  
point_picking_hostname       Computer used for point picking  
point_picking_comment       User comment for the point picking  
point_picking_write_time     time.processing UTC Timespamp of last point picking  
shutter_inversion bool     Shutter open when should be close and vice versa? (True / False)  
astrometry_number_stars       Number of stars used for calibration  
astrometry_calibration_image       Image used for calibration  
astrometry_catalog_efficiency       Percentage of non-spurious stars in the catalog  
astrometry_global_correction       Fine astrometric corrections applied  
astrometry_catalog     meta.software Catalog used for astrometry (actual catalog or image)  
astrometry_method     meta.software Calibration method (local, global…) .
photometry_raw_software     meta.software Software used to compute raw photometry  
photometry_raw_method     meta.software Meteor photometry method  
photometry_raw_write_time     time.processing UTC Timespamp of photometry calculation  
photometric_zero_point       photometric zero-point  
photometric_zero_point_uncertainty       $1-\sigma$ uncertainty of the zero-point  
photometric_zero_point_datetime       Date of the image in which photometric zero-point was calculated  
astrometry_write_time ISO8601   time.processing UTC Timespamp of conversion from pixels to sky  
triangulation_method     meta.software Triangulation method  
triangulation_write_time ISO8601   time.processing UTC Timespamp of triangulation runtime  
triangulation_file_N       observation files used for triangulation $N \in {2,\infty}$  
trajectory_analysis_velocity_order       Order of computation of velocities (1 = consecutive observations, 2 = starts together and ends together, $> 2$ = undefined)  
triangulation_ra_ecef_inf decimal deg   RA of radiant at infinity in ECEF frame  
triangulation_ra_ecef_inf_err decimal deg   error in RA of radiant at infinity in ECEF frame  
triangulation_dec_ecef_inf decimal deg   Dec of radiant at infinity in ECEF frame  
triangulation_dec_ecef_inf_err decimal deg   error in Dec of radiant at infinity in ECEF frame  
triangulation_ra_eci_inf decimal deg   RA of radiant at infinity in ECI frame  
triangulation_ra_eci_inf_err decimal deg   error in RA of radiant at infinity in ECI frame  
triangulation_dec_eci_inf decimal deg   Dec of radiant at infinity in ECI frame  
triangulation_dec_eci_inf_err decimal deg   error in Dec of radiant at infinity in ECI frame  
triangulation_slope_inf decimal deg   Slope of the radiant at infinity in ENU frame  
triangulation_slope_inf_err decimal deg   error in slope of the radiant at infinity in ENU frame  
triangulation_azimuth_inf decimal deg   Azimuth of the radiant at infinity in ENU frame  
triangulation_azimuth_inf_err decimal deg   error in Azimuth of the radiant at infinity in ENU frame  
ballistic_alpha       dimensionless ballistic parameter $\alpha$  
ballistic_beta       dimensionless mass loss parameter $\beta$  
ballistic_entry_mass   kg   entry mass calculated using MGs ballistic parameters given initially spherical body of density 3500kg/m3  
ballistic_alpha_all       dimensionless ballistic parameter $\alpha$ calculated using all cameras  
ballistic_beta_all       dimensionless mass loss parameter $\beta$ calculated using all cameras  
ballistic_entry_mass_all   kg   entry mass calculated using MGs ballistic parameters from all cameras given initially spherical body of density 3500kg/m3  
ballistic_reference_velocity   $\mbox{m s}^{-1}$   v0 used in Gritsevich fitting to normalise velocities  
EKS_smoother_run bool     True/False  
EKS_initial_velocity_all_cam   $\mbox{m s}^{-1}$   Entry velocity calculated using multiple cameras. Not part of pipeline. May be run on chosen cameras.  
EKS_initial_velocity_all_cam_err   $\mbox{m s}^{-1}$   1 sigma error on entry velocity calculated using multiple cameras within EKS.  

Columns

Show Table
Column Name Format Unit UCD Comment
__Point picking__        
datetime ISO8601   time.epoch Time expressed in UTC timescale
JD   day time.epoch Julian Day
time_err_plus   sec    
time_err_minus   sec    
x_image   pix   Physical X coordinate. Origin: left
y_image   pix   Physical Y coordinate. Origin: top
err_plus_x_image   pix    
err_plus_y_image   pix    
err_minus_x_image   pix    
err_minus_y_image   pix    
x_fits   pix   Physical X coordinate in the FITS standard (Y inversed)
y_fits   pix   Physical Y coordinate in the FITS standard (Y inversed)
err_plus_x_fits   pix    
err_plus_y_fits   pix    
err_minus_x_fits   pix    
err_minus_y_fits   pix    
de_bruijn_sequence_element_index       index in the De Bruijn sequence
dash_start_end       ‘start’ or ‘end’
periodic_pulse       periodic pulse is for future PFSK data
pick_flag       ‘M’: for manually picked
encoding_type       ‘PW’: pulse width, ‘PF’: pulse frequency
__Brightness__        
brightness_dash_V   ADU   Integrated brightness over dash in V band
err_plus_brightness_dash_V   ADU    
err_minus_brightness_dash_V   ADU    
flux_dash_V   ADU   Integrated brightness over dash in V band, normalised to exposure time
err_plus_flux_dash_V   ADU    
err_minus_flux_dash_V   ADU    
m_V   mag   apparent magnitude in V band
err_plus_m_V   mag    
err_minus_m_V   mag    
M_V   mag   absolute magnitude in V band (zero point @100km altitude)
err_plus_M_V   mag    
err_minus_M_V   mag    
__Calibration__        
azimuth decimal deg pos.az.azi East of North azimuth
err_plus_azimuth decimal deg pos.az.azi  
err_minus_azimuth decimal deg pos.az.azi  
altitude decimal deg pos.az.alt Angle above horizon (geometric, not apparent)
err_plus_altitude decimal deg pos.az.alt  
err_minus_altitude decimal deg pos.az.alt  
zenith_angle decimal deg pos.az.zd Angle from zenith
err_plus_zenith_angle decimal deg pos.az.zd  
err_minus_zenith_angle decimal deg pos.az.zd  

Misc

Event codenames

Events are automatically assigned a codename, based on the UTC date they happened at, following the format DNYYMMDD_SS:

Event folder structure

This is an example of a typical GFO event file strucure:

DN181225_03
├── 033_Ait_Ben_Haddou | system-number_camera-location
│   ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment_altaz.png | plot of alt/az coordinates in
                                    celestial sphere (from astrometric file)
│   ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment.ecsv | astrometric file
│   ├── 033_2018-12-25_183159_E_DSC_0064-G.fits | fits conversion of the fireball image
│   ├── 033_2018-12-25_183159_E_DSC_0064.NEF | raw fireball image
│   ├── 033_2018-12-25_183159_E_DSC_0064.thumb.2400.400.tile.jpg | detection tile from the camera (usually 400x400 greyscale image)
│   ├── 033_2018-12-25_183159_E_DSC_0064.thumb.jpg | jpeg colour extract of the fireball image 
                                    (warning: it is not exactly the same size the the RAW or FITS image)
│   ├── 2018-12-25_DFNEXT033_log_interval.txt | camera capture control log file
│   ├── dfnstation.cfg | camera station file
│   ├── calib | folder containing calibration data
│   │   ├── 033_2018-12-25_193029_E_DSC_0181-G_fh.fits | calibrated image (FITS header contains calibration coefficients)
│   │   ├── 033_2018-12-25_193029_E_DSC_0181-G_fh.matched_stars | list of matched stars
│   │   ├── 033_2018-12-25_193029_E_DSC_0181.NEF | raw calibration image
│   │   └── dfnstation.cfg | camera station file
│   └
│
├── eventflags.cfg | config file for all manual entries (event nickname, search status...)
│
├── WRF/
│   ├── profile_DN151127_01_1045.csv | 1D atmospheric profile centered on last astrometric point
│   ├── profile_DN151127_01_1045.pdf | plot of the above file
│   └── wrfout_d04_2015-11-27_06:00:00 | full 3D WRF atmospheric model 
│   
├── trajectory_auto_20190108 | automatic trajectory folder generated
│   ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment.ecsv | astrometric file
│   ├── 033_2018-12-25_183159_E_DSC_0064-G_DN181225_03_2019-01-07_111913_hadry_nocomment_point_picking_plot.jpg 
                                    | coulour plot of centroids on the image
│   ├── 2019-01-08_DFN_WS_log_orbit.txt | log file for the orbit calculation
│   ├── 2019-01-08_DFN_WS_log_trajectory.txt | log file for the trajectory determination and brightflight analysis
│   ├── 45_2018-12-25_183158_S_DSC_0058-G_DN181225_03_2019-01-07_111647_hadry_nocomment.ecsv
│   ├── 45_2018-12-25_183158_S_DSC_0058-G_DN181225_03_2019-01-07_111647_hadry_nocomment_point_picking_plot.jpg
│   ├── DN181225_03_key_parameters.yaml | key parameters for the event
│   ├── DN181225_03_MC_orbits.csv | Monte Carlo orbits calculated
│   ├── DN181225_03_MC_orbits.pdf | ecliptic orbital plot
│   └── DN181225_03_trajectory.kmz | Google Earth KMZ trajectory file
└── 

Firmware versions