It does sound very much like a prediction market thing rather than weather
research goal. While high-time resolution data (meaning that you have data at
frequent intervals) can have very valid research purposes , latency of a few
seconds outside of perhaps something like a production airport wind shear alert
system isn’t generally a problem for research.
For those unfamiliar, this is not the type of prediction market where you are
trying to forecast future commodity (grain, oil, energy, etc.) based on near
term weather conditions such as the price of soybeans depending on growing
season rain or the demand for natural gas based on daily temperature trends. A
few minutes of latency of real-time observations doesn’t impact those. The is
something like the Polymarket platform that’s a bit more like sports betting,
except on any sort of factual future event. (election outcomes, award winners
(e.g. Oscars), and even what the high temperature or rainfall total is going
to be for a given day in a location) In the context of weather, if you can
get real-time data a few seconds before your competition you can place last
minute bets to your advantage. The exchange of funds is done in
cryptocurrency. Unlike real markets, it is entirely about speculation and
produces little direct economic value because you’re not really trading
something of value but placing bets.
From: ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Ryan Hickman
Sent: Wednesday, March 18, 2026 11:29 AM
To: Thomas B <thms_brgg@xxxxxxxxx>
Cc: Charles Concodora <concodcw@xxxxxxxxxxxxxxxxx>; ldm-users@xxxxxxxxxxxxxxxx
Subject: Re: [ldm-users] Looking for upstream IDD feed for personal METAR
research project
Thomas,
I do have to say that this sounds very much like a prediction market request;
I've received over 20 of these in the last six months. I have 3 from the last
24 hours in my inbox, all sounding eerily similar to this.
EMWIN requires a satellite dish. The FTP is for archive purposes and is the
same as the TGFTP endpoint.
https://www.weather.gov/emwin/<https://urldefense.com/v3/__https:/www.weather.gov/emwin/__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXkSzxkJvw$>
You'll likely receive better answers to your Synoptic Data questions by asking
them directly.
On Wed, Mar 18, 2026 at 9:29 AM Thomas B
<thms_brgg@xxxxxxxxx<mailto:thms_brgg@xxxxxxxxx>> wrote:
Gregg,
Thank you very much for this detailed explanation of the METAR data flow — this
is incredibly helpful and aligns with what we've observed from benchmarking
multiple sources.
We've been polling TGFTP at 1-second intervals with If-Modified-Since headers
and it is indeed our fastest source, consistent with your assessment. We looked
into LDM/IDD through Unidata but were unable to obtain access, and SBN/NOAAPORT
satellite ingest is more hardware investment than we can justify at this stage.
Two questions if you don't mind:
1. EMWIN terrestrial server — since it's push-based, it could shave off the
polling latency we have on TGFTP. Could you point us toward the hostname/port
of the terrestrial EMWIN server, and whether it carries METAR from all ASOS
sites? Any documentation on the connection protocol would be greatly
appreciated.
2. SynopticData's FAA agreement — we use the Synoptic API and it sometimes
detects new observations before TGFTP in our benchmarks. Do you know if their
FAA connection gives them a tap upstream of NWSTG/GATEWAY, or is it the same
pipeline?
Best regards,
Le mardi 17 mars 2026 à 18:15, Gregory Grosshans
<gregory.grosshans@xxxxxxxx<mailto:gregory.grosshans@xxxxxxxx>> a écrit :
At a high level the flow of METAR data from ASOS sites at U.S. Airports
traverse the FAA networks to the NWS NWSTG/GATEWAY system. Once at
NWSTG/GATEWAY, essentially, the data is sent to
TGFTP.nws.noaa.gov<https://urldefense.com/v3/__http:/TGFTP.nws.noaa.gov__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXkQbbaFbg$>
as well as to the NCF, where the NCF uplinks the data to the SBN/NOAAPORT.
UNIDATA and other top-tier LDM/IDD sites with an SBN/NOAAPORT ingest system
will receive METAR data from the SBN/NOAAPORT and inject it into the LDM/IDD
network. Returning to the NWSTG/GATEWAY, they also send METAR data to other WMO
member countries at the same time they send the data to TGFTP and NCF.
MADIS is different from the NWSTG/GATEWAY. Note MADIS originated on the
research side of NOAA (i.e. OAR ESRL/GSD per your weblink) and then a version
became operational at NWS/NCEP many years ago. MADIS also collected various
mesonets. From my understanding MADIS is no longer being developed, instead the
NWS is utilizing SynopticData
<https://urldefense.com/v3/__https:/synopticdata.com/__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXkaqfcmRQ$>
to acquire the various mesonets. I understand SynopticData also connects with
the FAA to acquire observational data via a special agreement. You would have
to check with the FAA (and I'm not sure who it would be) to see if other
private sector companies connect to the FAA to receive METAR data.
I suspect the lowest latency will be obtaining data from TGFTP, followed by an
LDM feed from the IDD.
Some weather enthusiasts repurpose old satellite dishes from the 1980s or
1990s, originally used for satellite TV, for their own SBN/NOAAPORT satellite
ingest systems. This includes buying a NOVRA box, computer, etc to ingest the
data from the dish. This is an option to obtain METAR data from the SBN and it
would be slightly faster than the latency introduced by the IDD (which is
likely only a few seconds faster depending on how close your connection is to a
top-tier site). You could also set up a smaller satellite dish (compared to the
SBN/NOAAPORT dish) and use the Emergency Managers Weather Information Network
(EMWIN), which is supposed to include METAR observations. I suspect the route
of METAR data for EMWIN goes from NWSTG/GATEWAY -> NESDIS -> GOES EAST/WEST
satellites. Also, I believe there is a terrestrial based EMWIN server if you
don't want to set up a satellite ingest system.
Gregg
On Tue, Mar 17, 2026 at 10:06 AM Thomas B
<thms_brgg@xxxxxxxxx<mailto:thms_brgg@xxxxxxxxx>> wrote:
Hi,
Thanks for the pointer to
tgftp.nws.noaa.gov<https://urldefense.com/v3/__http:/tgftp.nws.noaa.gov__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXnvu25uoA$>
— I'm currently polling it and it does seem to be one of the fastest publicly
available HTTP sources for METAR.
However, from what I've been reading, tgftp serves static files that are
regenerated on a cycle (the MADIS documentation at
https://madis.ncep.noaa.gov/madis_metar.shtml<https://urldefense.com/v3/__https:/madis.ncep.noaa.gov/madis_metar.shtml__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXnbQtNQNA$>
mentions data is "processed every 5 minutes"). So even with aggressive
polling, there's an inherent delay of up to several minutes between the
observation time and when it appears on tgftp.
By contrast, the LDM/IDD network distributes METAR via push as soon as it's
injected from NOAAPort/SBN. The LDM network troubleshooting docs
(https://docs.unidata.ucar.edu/ldm/current/troubleshooting/networkTrouble.html<https://urldefense.com/v3/__https:/docs.unidata.ucar.edu/ldm/current/troubleshooting/networkTrouble.html__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXmZRhF_rg$>)
reference sub-second product latency as typical for well-connected IDD nodes.
For my use case, that difference matters a lot — I need the lowest possible
latency on METAR observations. So I'm really interested in getting an LDM feed
rather than polling tgftp.
I saw in the FAQ that non-academic users can sometimes arrange a feed from a
willing upstream participant.
Thanks again for your help.
Le lundi 16 mars 2026 à 22:26, Charles Concodora
<concodcw@xxxxxxxxxxxxxxxxx<mailto:concodcw@xxxxxxxxxxxxxxxxx>> a écrit :
As far as I'm aware,
tgftp.nws.noaa.gov<https://urldefense.com/v3/__http:/tgftp.nws.noaa.gov/__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXl7t4hceQ$>
has the lowest latency.
On Mar 15, 2026, at 2:30 PM, Thomas B
<thms_brgg@xxxxxxxxx<mailto:thms_brgg@xxxxxxxxx>> wrote:
tgftp.nws.noaa.gov<https://urldefense.com/v3/__http:/tgftp.nws.noaa.gov/__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXl7t4hceQ$>
_________________________________________________________
NOTE: All exchanges posted to NSF Unidata maintained email lists are
made publicly available through the web. Users who post to any of the
lists we maintain are reminded to remove any personal information that
they do not want to be made public.
NSF Unidata ldm-users Mailing List
(ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>)
For list information, to unsubscribe, or change your membership options,
visit:
https://mailinglists.unidata.ucar.edu/listinfo/ldm-users/<https://urldefense.com/v3/__https:/mailinglists.unidata.ucar.edu/listinfo/ldm-users/__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXm4V4NxMg$>
--
********************
========================================================================
Email seems to be generating increasing inefficiencies in organizations. I
learned from a manager a Stanford Computer Science professor no longer uses
email for communication, but uses SNAIL mail, telephone calls, and person to
person visits. I'm considering the same.
405-325-2462
Storm Prediction Center
120 David L. Boren Blvd, Suite 2330
Norman, OK 73072
_________________________________________________________
NOTE: All exchanges posted to NSF Unidata maintained email lists are
made publicly available through the web. Users who post to any of the
lists we maintain are reminded to remove any personal information that
they do not want to be made public.
NSF Unidata ldm-users Mailing List
(ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>)
For list information, to unsubscribe, or change your membership options,
visit:
https://mailinglists.unidata.ucar.edu/listinfo/ldm-users/<https://urldefense.com/v3/__https:/mailinglists.unidata.ucar.edu/listinfo/ldm-users/__;!!DZ3fjg!-CTZyrT7jefIfO_ai82m-O_lkQewOcB1pyKhY4ZU6nWdHAmQK6SRw9NfL5Wfy-OVMFDJ3d9UDXm4V4NxMg$>