[This local archive copy mirrored from the canonical site: http://zowie.metnet.navy.mil/~spawar/JMV-TNG/XML/OMF.html; links may not have complete integrity, so use the canonical document at this URL if possible.]

Weather Observation Definition Format (OMF)

Motivation
Structure of an OMF document
1. Basic attributes
2. Reports Element
SIGMET advisories
Example
Post-processing of OMF documents for JMV
1. Computing of Flight Categories
References (bibliography)

Motivation

Weather Observation Markup Format is an application of XML to describe a particular kind of documents: weather observation reports.

Weather observations are reported in a variety of formats: FM 15-X Ext. METAR, FM 16-X Ext. SPECI, FM 51-X Ext. TAF, etc. These reports constitute KAWN, WMO etc. feeds. Unfortunately, these formats are rather unsuitable for serving of observation information requested for a particular area of interest:

The reports refer to observation stations only by their call-sign or block-id. A report itself does not specify station's location or its full name. Every entity that interprets the reports is supposed to possess a current copy of a Master Station library, which contains such static information about the stations. Although this arrangement is well suited for processing of a massive feed, it is not convenient for a client that is only interested in weather reports for a (relatively) small area. In this case, the description of an observation station ought to be carried along with its report, as an annotation.
The standard KAWN, WMO reports do not bear the complete timestamps: the time of an observation is recorded only in HHMM or DDHHMM format, omitting the month and the year. This makes the raw reports unsuitable for archival storage, record keeping, trend processing, etc.
Report consumers are frequently interested in derived quantities like cloud ceiling, flight conditions, or humidity. These parameters can be computed from the data in the report; however, the computation is sometimes not trivial, and often messy. It makes sense therefore for a server to compute the most popular derived parameters and serve them along with the raw report data.

The intent of OMF therefore is annotation of weather reports. The reports should be distributed as they are without any mangling. On the other hand, we would like to mark them up with station information (location, name), and with a few derived parameters. This document is to define the format of this markup.

OMF is an application of XML, and by its virtue, an application of SGML. SGML is used extensively within DoD for documenting of various types of information (military standards, procurement materials, service manuals). OMF brings weather observations into the same fold.

Thus, the design goals of OMF are:

Mark up (annotate) raw observation reports with additional description and derived, computed quantities.
The raw report data must not be modified in any way, and should be conveniently extractable (by simply stripping all the tags away).
OMF must be concise. While providing useful annotations to a client, OMF markup should not impose undue overhead on communication channels.
It should be possible to extend the markup with additional annotations, without affecting applications that do not use this information.

Structure of an OMF document

The Observation Markup Format is an application of the Extensible Markup Language (XML). Thus its definition consists of the various XML elements defined in this section. See the references section for information on XML for further details.

The OMF contains the following elements:

Reports - defines a group of weather reports
- METAR for a single METAR report
- SPECI for a single SPECI report
Advisories - defines a collection of weather hazard warnings
- SIGMET - SIGnificant METeorological Information

The following sections define the major elements along with the minor elements that are relevant to them. In each section, XML DTD declarations are provided for precise definition of elements and attributes. The collection of XML DTD declarations found in this specification can be arbitrarily extended to add new elements and attributes for new enhancements.

Some of elements attributes are common. For compactness, they are defined in the following special section.

Basic Attributes

The following attributes provide basic annotation information. These attributes may (and often must) be present in appropriate elements describing reports.

TStamp
Time Stamp unsigned integer number
UTC time in seconds since the Epoch, 00:00:00 Jan 1, 1970 UTC. This is the value returned by a POSIX function time(2).

LatLon
Specification of a point on the globe a string of a form "aaa.bbb, ccc.ddd"
where aaa.bbb and ccc.ddd are signed floating point numbers telling the latitude resp. longitude of a point on the globe, in whole and fractional degrees.
The numbers are positive for Northern latitudes resp. Eastern longitudes, and negative otherwise.
The range of the numbers is [-90.0, 90.0] for latitudes, (-180.0, 180.0] for longitudes.

LatLons
Specification of a sequence of points on the globe a string of a form "lat1, lon1, lat2, lon2, latn, lonn"
where lat1, lon1, etc. are signed floating point numbers telling the latitude resp. longitude of a point on the globe, in whole and fractional degrees.
See a LatLon attribute above for more details.

BBox
Bounding box, which tells the latitudal and the longitudal spans of an area of the globe a string of a form "lat-N, lon-W, lat-S, lon-E"
Here lat-N is the latitude of the Northern-most point of the area, lat-S is the latitude of the Southern-most point, lon-W is the longitude of the Western-most point of the area, and lon-E is the Eastern-most longitude.
lats and lons are signed floating-point numbers, in degrees. See a LatLon attribute above for more details.
It is required that lat-N >= lat-S. The left-lon (lon-W) may however be greater than the right-lon (lon-E). For example, a range of longitudes [-170,170] specifies the entire world but Indonesia. On the other end, the range [170,-170] includes Indonesia only. By the same token, [-10,10] pertains to a 21-degree longitude strip along the Grinwitch meridian, while [10,-10] specifies the whole globe but that strip.

BId
Block id of a weather observation station (WMO number) unsigned integer number

SName
Call sign and the full name of a weather observation station a string of the form "ccccc, name"
where ccccc are the call letters of the station (ICAO station id: 4 or 5 upper-case letters, may be omitted), name is an arbitrary string describing the station

`TStamp`
	Time Stamp	unsigned integer number
	UTC time in seconds since the Epoch, `00:00:00 Jan 1, 1970 UTC`. This is the value returned by a POSIX function `time(2)`.

`LatLon`
	Specification of a point on the globe	a string of a form `"aaa.bbb, ccc.ddd"`
	where `aaa.bbb` and `ccc.ddd` are signed floating point numbers telling the latitude resp. longitude of a point on the globe, in whole and fractional degrees. The numbers are positive for Northern latitudes resp. Eastern longitudes, and negative otherwise. The range of the numbers is [-90.0, 90.0] for latitudes, (-180.0, 180.0] for longitudes.

`LatLons`
	Specification of a sequence of points on the globe	a string of a form `"lat1, lon1, lat2, lon2, latn, lonn"`
	where `lat1`, `lon1`, etc. are signed floating point numbers telling the latitude resp. longitude of a point on the globe, in whole and fractional degrees. See a `LatLon` attribute above for more details.

`BBox`
	Bounding box, which tells the latitudal and the longitudal spans of an area of the globe	a string of a form `"lat-N, lon-W, lat-S, lon-E"`
	Here `lat-N` is the latitude of the Northern-most point of the area, `lat-S` is the latitude of the Southern-most point, `lon-W` is the longitude of the Western-most point of the area, and `lon-E` is the Eastern-most longitude. `lat`s and `lon`s are signed floating-point numbers, in degrees. See a `LatLon` attribute above for more details. It is required that `lat-N` >= `lat-S`. The left-lon (`lon-W`) may however be greater than the right-lon (`lon-E`). For example, a range of longitudes `[-170,170]` specifies the entire world but Indonesia. On the other end, the range `[170,-170]` includes Indonesia only. By the same token, `[-10,10]` pertains to a 21-degree longitude strip along the Grinwitch meridian, while `[10,-10]` specifies the whole globe but that strip.

`BId`
	Block id of a weather observation station (WMO number)	unsigned integer number

`SName`
	Call sign and the full name of a weather observation station	a string of the form `"ccccc, name"`
	where `ccccc` are the call letters of the station (`ICAO` station id: 4 or 5 upper-case letters, may be omitted), `name` is an arbitrary string describing the station

XML DTD

<!ENTITY % TStamp-type "NMTOKEN">
<!ENTITY % TStamp "TStamp %TStamp-type; #REQUIRED">
<!ENTITY % LatLon "LatLon CDATA #REQUIRED">
<!ENTITY % LatLons "LatLons CDATA #REQUIRED">
<!ENTITY % BBox-REQD "BBox CDATA #REQUIRED">
<!ENTITY % BBox-OPT "BBox CDATA #IMPLIED">
<!ENTITY % BId "BId NMTOKEN #REQUIRED">
<!ENTITY % SName "SName CDATA #REQUIRED">

`Reports` Element

Defines a collection of weather observation reports. The collection is made of METAR and SPECI elements, each defining one particular report.

XML DTD

<!ELEMENT Reports ( METAR | SPECI )* >

Attributes

`TStamp`
	Time Stamp when this group of reports was created. Normally it's the time the request is fulfilled.

<!ATTLIST Reports %TStamp;>

Common report attributes

The following attributes are shared among various report elements. These attributes define report's annotation.

TStamp
Time Stamp when the station reported an observation

LatLon
Location of the station

BId
Block ID of the reporting station

SName
The station's call letters and the full name, if any

Vis
Horizontal Visibility a number of meters, omitted, or a special token "INF"
The Vis attribute is always given in meters, no matter how the corresponding token is specified in the report: as a number of meters, km, whole or fractional statutory miles.
Vis is set to "INF" if the corresponding token in the report is 9999 or CAVOK. If visibility is reported as less than 1/4SM, the Vis attribute is set to "0".

Ceiling
Cloud Ceiling a number of feet, omitted, or a special token "INF"
The attribute is set based on a number of report tokens, as the following algorithm outlines:

if a VVxxx token is present, the Ceiling is set to be 100 * atoi(xxx)
if BKNxxx or OVCxxx tokens are present, Ceiling is 100 * atoi(xxx) for the first BKNxxx or OVCxxx token occurring in the report
if only FEWxxx or SCTxxx tokens are present, set Ceiling to "INF"
if SKC, CLR or CAVOK are present, Ceiling is set to "INF"
otherwise, the cloud ceiling is unknown and omitted

`TStamp`
	Time Stamp when the station reported an observation

`LatLon`
	Location of the station

`BId`
	Block ID of the reporting station

`SName`
	The station's call letters and the full name, if any

`Vis`
	Horizontal Visibility	a number of meters, omitted, or a special token "`INF`"
	The `Vis` attribute is always given in meters, no matter how the corresponding token is specified in the report: as a number of meters, km, whole or fractional statutory miles. `Vis` is set to "`INF`" if the corresponding token in the report is 9999 or `CAVOK`. If visibility is reported as less than `1/4SM`, the `Vis` attribute is set to "0".
`Ceiling`
	Cloud Ceiling	a number of feet, omitted, or a special token "`INF`"
	The attribute is set based on a number of report tokens, as the following algorithm outlines: if a `VVxxx` token is present, the `Ceiling` is set to be `100 * atoi(xxx)` if `BKNxxx` or `OVCxxx` tokens are present, `Ceiling` is `100 * atoi(xxx)` for the first `BKNxxx` or `OVCxxx` token occurring in the report if only `FEWxxx` or `SCTxxx` tokens are present, set `Ceiling` to "`INF`" if `SKC`, `CLR` or `CAVOK` are present, `Ceiling` is set to "`INF`" otherwise, the cloud ceiling is unknown and omitted

XML DTD

<!ENTITY % ReportAttrs 
    "%TStamp; %LatLon; %BId; %SName;
     Vis NMTOKEN  #IMPLIED
     Ceiling NMTOKEN  #IMPLIED
    ">

`METAR` Report

This element defines one particular METAR report from one particular station. The element's content is the report itself (a collection of tokens), exactly as it was received from the feed. The element's attributes annotate the report.

XML DTD

<!ELEMENT METAR (#PCDATA)>
<!ATTLIST METAR %ReportAttrs;>

`SPECI` Report

This element defines one particular SPECI report from one particular station. The element's content is the report itself (a collection of tokens), exactly as it was received from the feed. The element's attributes annotate the report.

XML DTD

<!ELEMENT SPECI (#PCDATA)>
<!ATTLIST SPECI %ReportAttrs;>

OMF Example

<!DOCTYPE OMF SYSTEM "http://zowie.metnet.navy.mil/~spawar/JMV-TNG/XML/OMF.dtd">
<Reports TStamp="888965153">
<METAR TStamp="888965153" LatLon="36.58, -121.85" BId="724915"
   SName="KMRY, MONTEREY PENINSULA" Vis="80500" Ceiling="INF">
KMRY 032245Z 29007KT 50SM SKC 15/03 A3003</METAR>
<SPECI TStamp="888966299" LatLon="36.97, -86.42" BId="746716"
   SName="KBWG, BOWLING GREEN" Vis="12880" Ceiling="2400">
KBWG 032304Z VRB05KT 8SM BKN024 OVC035 04/01 A2990 RMK AO2</SPECI>
<METAR TStamp="888968939" LatLon="37.00, -101.9" BId="724604"
   SName="KEHA, ELKHART (AWOS)">
KEHA 032348Z AUTO 08004KT 15/M10 RMK AO1 PK WND 06 000 T01501099</METAR>
<METAR TStamp="888967859" LatLon="36.67, -4.48" BId="84820"
   SName="LEMG, MALAGA (CIV/MIL)" Vis="4000" Ceiling="INF">
LEMG 2330Z 31006KT 4000 BR FEW008 08/07 Q1027 NOSIG</METAR>
</Reports>

For comparison, the following is the same information in the old SYN format. Note that despite annotations, OMF is compact:

   KMRY  36.6 -121.8***************************************01
3 METAR 3658 -12185 888965153 888972353 724915 KMRY MONTEREY PENINSULA
METAR KMRY 032245Z 29007KT 50SM SKC 15/03 A3003
TAF NOT AVAILABLE
   KBWG  37.0  -86.4***************************************02
4 SPECI 3697 -8642 888966299 888973499 746716 KBWG BOWLING GREEN
SPECI KBWG 032304Z VRB05KT 8SM BKN024 OVC035 04/01 A2990
RMK AO2
TAF NOT AVAILABLE
   KEHA  37.0 -101.9***************************************00
4 METAR 3700 -10190 888968939 888976139 724604 KEHA ELKHART (AWOS)
METAR KEHA 032348Z AUTO 08004KT 15/M10 RMK AO1 PK WND 06
000 T01501099
TAF NOT AVAILABLE
   LEMG  36.7   -4.5***************************************03
3 METAR 3667 -448 888967859 888975059 84820 LEMG MALAGA (CIV/MIL)
METAR LEMG 2330Z 31006KT 4000 BR FEW008 08/07 Q1027 NOSIG
TAF NOT AVAILABLE

Post-processing of OMF documents for JMV

JMV stores weather reports received for a specific area in a "database", which can be described as follows

CREATE TABLE Observations (
block_id INTEGER not null, Station's block id
ref_time DATETIME YEAR TO SECOND not null, Timestamp of the report
latlon VARCHAR(16) not null,
station_name VARCHAR(40),
visibility VARCHAR(5),
ceiling VARCHAR(5),
tokens, VARCHAR(255,61) not null Tokens of the actual report, with METAR or SPECI prepended
primary key (block_id)
);

This table is actually stored in a flat file (for example, MZZUPDT_.XXF file), one row per line, with fields delimited with a character "|". There is one extra character "|" at the end of the line, right after the last field and before the end-of-line character(s).

For example, the OMF document above would be stored in the MZZUPDT_.XXF file as follows:

724915|888965153|36.58, -121.85|KMRY, MONTEREY PENINSULA|80500|INF|METAR KMRY 032245Z 29007KT 50SM SKC 15/03 A3003|
746716|888966299|36.97, -86.42|KBWG, BOWLING GREEN|12880|2400|SPECI KBWG 032304Z VRB05KT 8SM BKN024 OVC035 04/01 A2990 RMK AO2|
724604|888968939|37.00, -101.9|KEHA, ELKHART (AWOS)|||METAR KEHA 032348Z AUTO 08004KT 15/M10 RMK AO1 PK WND 06 000 T01501099|
84820|888967859|36.67, -4.48|LEMG, MALAGA (CIV/MIL)|4000|INF|METAR LEMG 2330Z 31006KT 4000 BR FEW008 08/07 Q1027 NOSIG|

When a new OMF document with updated reports for the same area of interest arrives, a text/x-omf helper, syn-proc, merges the new information with the one already contained in the local database, MZZUPDT_.XXF. Specifically, the expired observations (the ones that are past their valid time) are purged, and a new report of an observation station replaces the old one if existed. The helper thus reinforces a primary key constrain: each record in the database has a unique block_id.

Computing of Flight Categories

Horizontal visibility and the cloud ceiling can be used to compute flight categories. The algorithm outlined below was explained to me by LCDR Mike Neith.

A flight category is determined by visibility and cloud ceiling. First, we need to quantify these parameters: convert their numerical values into symbols, by simple thresholding:

Quantifying the reported prevailing visibility
visibility <= 4830 meters (3 miles) ---> visibility_symbol = IFR
visibility <= 8050 meters (5 miles) ---> visibility_symbol = MVFR
visibility > 8050 meters (5 miles) or "INF" ---> visibility_symbol = VFR
visibility was not reported ---> visibility_symbol = unknown

Quantifying the ceiling, based on reported cloud conditions:
ceiling <= 1000 (feet) ---> ceiling_symbol = IFR
ceiling <= 3000 (feet) ---> ceiling_symbol = MVFR
ceiling > 3000 (feet) or "INF" ---> ceiling_symbol = VFR
ceiling is omitted ---> ceiling_symbol = unknown

Flight category is then, informally, the "smallest" of the two symbols, the visibility_symbol and the ceiling_symbol. To be more precise:

FlightCategory is:
if
visibility_symbol is IFR or ceiling_symbol IFR ---> IFR
else if
visibility_symbol is MVFR or ceiling_symbol MVFR ---> MVFR
else if
visibility_symbol is VFR or ceiling_symbol VFR ---> VFR
else
(that is, both symbols are unknown) ---> unknown

References

Extensible Markup Language (XML) 1.0. W3C Recommendation 10-February-1998 http://www.w3.org/TR/1998/REC-xml-19980210

The XML FAQ
Frequently Asked Questions about the Extensible Markup Language

Extensible Markup Language (XML), a part of the SGML/XML Web Page
http://www.sil.org/sgml/xml.html
This site contains extensive references to XML examples, non-examples, and available software.

XML Weather Station Demo
An alternative (and rather primitive) way of describing weather conditions of particular cities. Still, some ideas are relevant and illustrate the points of the current proposal.

OMF DTD
The OMF Data Definition Document in XML format

Sample OMF document

This site's top page is http://zowie.metnet.navy.mil/~spawar/JMV-TNG/

oleg@pobox.com or oleg@acm.org
Your comments, problem reports, questions are very welcome!

Weather Observation Definition Format (OMF)

Motivation

Structure of an OMF document

Basic Attributes

XML DTD

`Reports` Element

XML DTD

Attributes

Common report attributes

XML DTD

`METAR` Report

XML DTD

`SPECI` Report

XML DTD

OMF Example

Post-processing of OMF documents for JMV

Computing of Flight Categories

References

History

$Id: OMF.html,v 1.5 1998/05/22 16:43:02 oleg Exp oleg $

`CREATE TABLE Observations (`
`block_id`	`INTEGER not null,`	Station's block id
`ref_time`	`DATETIME YEAR TO SECOND not null,`	Timestamp of the report
`latlon`	`VARCHAR(16) not null,`
`station_name`	`VARCHAR(40),`
`visibility`	`VARCHAR(5),`
`ceiling`	`VARCHAR(5),`
`tokens,`	`VARCHAR(255,61) not null`	Tokens of the actual report, with `METAR` or `SPECI` prepended
`primary key (block_id)`
`);`

Quantifying the reported prevailing visibility
visibility <= 4830 meters (3 miles)	--->	visibility_symbol = `IFR`
visibility <= 8050 meters (5 miles)	--->	visibility_symbol = `MVFR`
visibility > 8050 meters (5 miles) or "`INF`"	--->	visibility_symbol = `VFR`
visibility was not reported	--->	visibility_symbol = unknown

Quantifying the ceiling, based on reported cloud conditions:
ceiling <= 1000 (feet)	--->	ceiling_symbol = `IFR`
ceiling <= 3000 (feet)	--->	ceiling_symbol = `MVFR`
ceiling > 3000 (feet) or "`INF`"	--->	ceiling_symbol = `VFR`
ceiling is omitted	--->	ceiling_symbol = unknown

FlightCategory is:
`if`
	visibility_symbol is `IFR` or ceiling_symbol `IFR`	--->	`IFR`
`else if`
	visibility_symbol is `MVFR` or ceiling_symbol `MVFR`	--->	`MVFR`
`else if`
	visibility_symbol is `VFR` or ceiling_symbol `VFR`	--->	`VFR`
`else`
	(that is, both symbols are unknown)	--->	unknown

Weather Observation Definition Format (OMF)

Motivation

Structure of an OMF document

Basic Attributes

XML DTD

Reports Element

XML DTD

Attributes

Common report attributes

XML DTD

METAR Report

XML DTD

SPECI Report

XML DTD

OMF Example

Post-processing of OMF documents for JMV

Computing of Flight Categories

References

History

$Id: OMF.html,v 1.5 1998/05/22 16:43:02 oleg Exp oleg $

`Reports` Element

`METAR` Report

`SPECI` Report