Hi Gregg,
Thank you for all the detailed explanations — the pipeline diagram and the
points about collectives vs. individual METARs and the 5-minute ASOS
observations were really insightful. This gives me a much clearer picture of
how the data flows.
I think I have everything I need to move forward with my project. I really
appreciate you taking the time to help.
Best regards,
Le mercredi 18 mars 2026 à 15:10, Gregory Grosshans
<gregory.grosshans@xxxxxxxx> a écrit :
> Regarding polling TGFTP, keep in mind NCO has rate-limiting set up. I don't
> know the limit, for details you would need to inquire with NCO.
>
> For more details on EMWIN, including the terrestrial server please see:
> https://www.weather.gov/emwin/
> Access information and documents are available at this site. The EMWIN
> terrestrial server would be considered a pull-based system, and the EMWIN
> satellite solution would be considered a push-based system. I suggest
> interacting with other EMWIN users to learn more about the pros and cons of
> each option and timeliness. The satellite solution will remove multiple
> points of failure present in a terrestrial based network for data access.
>
> One way to view the FAA connections for METAR data acquisition is as follows:
>
> FAA--->NWSTG--------->TGFTP
> \-->SynopticData-->SyntopicDataAPI
>
> When comparing the timestamp on a METAR observation received from TGFTP
> versus the Synoptic API, is the actual METAR site comparison the same? That
> is, is the METAR obtained via TGFTP specific to that METAR site or was it
> part of a collective bulletin containing several/multiple METAR sites? If
> it's a collective, that may add some latency. I do not believe the FAA is
> creating the collectives, I think it is the NWS/NCO but you would need to
> check with NCO to confirm. Also, when comparing the timestamp, are you
> comparing the actual METAR observation from TGFTP to a METAR observation from
> SynopticData, or is it possible the SynopticData METAR is actually a 5-minute
> observation and not technically the METAR observation?
>
> Gregg
>
> On Wed, Mar 18, 2026 at 1:16 AM Thomas B <thms_brgg@xxxxxxxxx> wrote:
>
>> Gregg,
>>
>> Thank you very much for this detailed explanation of the METAR data flow —
>> this is incredibly helpful and aligns with what we've observed from
>> benchmarking multiple sources.
>>
>> We've been polling TGFTP at 1-second intervals with If-Modified-Since
>> headers and it is indeed our fastest source, consistent with your
>> assessment. We looked into LDM/IDD through Unidata but were unable to obtain
>> access, and SBN/NOAAPORT satellite ingest is more hardware investment than
>> we can justify at this stage.
>>
>> Two questions if you don't mind:
>>
>> 1. EMWIN terrestrial server — since it's push-based, it could shave off the
>> polling latency we have on TGFTP. Could you point us toward the
>> hostname/port of the terrestrial EMWIN server, and whether it carries METAR
>> from all ASOS sites? Any documentation on the connection protocol would be
>> greatly appreciated.
>> 2. SynopticData's FAA agreement — we use the Synoptic API and it sometimes
>> detects new observations before TGFTP in our benchmarks. Do you know if
>> their FAA connection gives them a tap upstream of NWSTG/GATEWAY, or is it
>> the same pipeline?
>> Best regards,
>> Le mardi 17 mars 2026 à 18:15, Gregory Grosshans
>> <gregory.grosshans@xxxxxxxx> a écrit :
>>
>>> At a high level the flow of METAR data from ASOS sites at U.S. Airports
>>> traverse the FAA networks to the NWS NWSTG/GATEWAY system. Once at
>>> NWSTG/GATEWAY, essentially, the data is sent to TGFTP.nws.noaa.gov as well
>>> as to the NCF, where the NCF uplinks the data to the SBN/NOAAPORT. UNIDATA
>>> and other top-tier LDM/IDD sites with an SBN/NOAAPORT ingest system will
>>> receive METAR data from the SBN/NOAAPORT and inject it into the LDM/IDD
>>> network. Returning to the NWSTG/GATEWAY, they also send METAR data to other
>>> WMO member countries at the same time they send the data to TGFTP and NCF.
>>>
>>> MADIS is different from the NWSTG/GATEWAY. Note MADIS originated on the
>>> research side of NOAA (i.e. OAR ESRL/GSD per your weblink) and then a
>>> version became operational at NWS/NCEP many years ago. MADIS also collected
>>> various mesonets. From my understanding MADIS is no longer being developed,
>>> instead the NWS is utilizing [SynopticData](https://synopticdata.com/)to
>>> acquire the various mesonets. I understand SynopticData also connects with
>>> the FAA to acquire observational data via a special agreement. You would
>>> have to check with the FAA (and I'm not sure who it would be) to see if
>>> other private sector companies connect to the FAA to receive METAR data.
>>>
>>> I suspect the lowest latency will be obtaining data from TGFTP, followed by
>>> an LDM feed from the IDD.
>>>
>>> Some weather enthusiasts repurpose old satellite dishes from the 1980s or
>>> 1990s, originally used for satellite TV, for their own SBN/NOAAPORT
>>> satellite ingest systems. This includes buying a NOVRA box, computer, etc
>>> to ingest the data from the dish. This is an option to obtain METAR data
>>> from the SBN and it would be slightly faster than the latency introduced by
>>> the IDD (which is likely only a few seconds faster depending on how close
>>> your connection is to a top-tier site). You could also set up a smaller
>>> satellite dish (compared to the SBN/NOAAPORT dish) and use the Emergency
>>> Managers Weather Information Network (EMWIN), which is supposed to include
>>> METAR observations. I suspect the route of METAR data for EMWIN goes from
>>> NWSTG/GATEWAY -> NESDIS -> GOES EAST/WEST satellites. Also, I believe there
>>> is a terrestrial based EMWIN server if you don't want to set up a satellite
>>> ingest system.
>>>
>>> Gregg
>>>
>>> On Tue, Mar 17, 2026 at 10:06 AM Thomas B <thms_brgg@xxxxxxxxx> wrote:
>>>
>>>> Hi,
>>>>
>>>> Thanks for the pointer to tgftp.nws.noaa.gov — I'm currently polling it
>>>> and it does seem to be one of the fastest publicly available HTTP sources
>>>> for METAR.
>>>>
>>>> However, from what I've been reading, tgftp serves static files that are
>>>> regenerated on a cycle (the MADIS documentation at
>>>> https://madis.ncep.noaa.gov/madis_metar.shtml mentions data is "processed
>>>> every 5 minutes"). So even with aggressive polling, there's an inherent
>>>> delay of up to several minutes between the observation time and when it
>>>> appears on tgftp.
>>>>
>>>> By contrast, the LDM/IDD network distributes METAR via push as soon as
>>>> it's injected from NOAAPort/SBN. The LDM network troubleshooting docs
>>>> (https://docs.unidata.ucar.edu/ldm/current/troubleshooting/networkTrouble.html)
>>>> reference sub-second product latency as typical for well-connected IDD
>>>> nodes.
>>>>
>>>> For my use case, that difference matters a lot — I need the lowest
>>>> possible latency on METAR observations. So I'm really interested in
>>>> getting an LDM feed rather than polling tgftp.
>>>>
>>>> I saw in the FAQ that non-academic users can sometimes arrange a feed from
>>>> a willing upstream participant.
>>>>
>>>> Thanks again for your help.
>>>> Le lundi 16 mars 2026 à 22:26, Charles Concodora
>>>> <concodcw@xxxxxxxxxxxxxxxxx> a écrit :
>>>>
>>>>> As far as I'm aware, [tgftp.nws.noaa.gov](http://tgftp.nws.noaa.gov/) has
>>>>> the lowest latency.
>>>>>
>>>>>> On Mar 15, 2026, at 2:30 PM, Thomas B <thms_brgg@xxxxxxxxx> wrote:
>>>>>>
>>>>>> [tgftp.nws.noaa.gov](http://tgftp.nws.noaa.gov/)
>>>>
>>>> _________________________________________________________
>>>> NOTE: All exchanges posted to NSF Unidata maintained email lists are
>>>> made publicly available through the web. Users who post to any of the
>>>> lists we maintain are reminded to remove any personal information that
>>>> they do not want to be made public.
>>>>
>>>> NSF Unidata ldm-users Mailing List
>>>> (ldm-users@xxxxxxxxxxxxxxxx)
>>>> For list information, to unsubscribe, or change your membership options,
>>>> visit: https://mailinglists.unidata.ucar.edu/listinfo/ldm-users/
>
> --
>
> ********************
> ========================================================================
> Email seems to be generating increasing inefficiencies in organizations. I
> learned from a manager a Stanford Computer Science professor no longer uses
> email for communication, but uses SNAIL mail, telephone calls, and person to
> person visits. I'm considering the same.
>
> 405-325-2462Storm Prediction Center120 David L. Boren Blvd, Suite 2330
> Norman, OK 73072