...with Dr. Jeff Copeland, Chief Scientist at WeatherFlow-Tempest. Jeff spoke to us from the WF-T Weather Lab in Fort Collins, Colorado, where his team of atmospheric and data scientists conducts forecast modeling to support the company’s suite of products and services.
Jeff relies on artificial intelligence (AI) and machine learning (ML) applications in his work and helped shed some light on how it’s leveraged at WeatherFlow-Tempest as one layer of their patented Nearcast™ Technology.
Q: CAN YOU EXPLAIN WHAT NEARCAST™ TECHNOLOGY IS AND HOW IT WORKS?
A: Nearcast™ Technology is the collection of behind-the-scenes data processing systems that makes extensive use of AI to deliver a high-value, curated dataset better informing us of what is happening now, what happened yesterday, and what will happen tomorrow. This dataset is used to drive improved decision support so that our customers can control the costs and other impacts of weather on them
Broken down into steps, we first gather weather data from the Tempest Network. This includes more than 50,000 Tempest stations and tens of thousands of other stations, combined with other sources such as satellites, radar, government, etc. Second, the data is curated using advanced quality control and scoring routines, qualifying the data for ingestion into our real-time data feed and forecast modeling systems. Lastly, the curated data are combined with the best available physics-based numerical forecast models to train our AI model. The final AI production models are run on rapid update cycles to produce the Tempest Forecasts, which are produced at all station locations and points between using our Weather Anywhere system. We then deliver the validated data via our API to powerful mobile and web applications and SaaS tools
Q: HOW WOULD YOU DEFINE AI IN THE SIMPLEST WAY POSSIBLE?
A: The best way to describe it is sophisticated pattern recognition. AI is good at finding patterns in complex data that humans or other simpler statistical techniques cannot spot. Once these patterns are known, they can be used to make new predictions or correct existing predictions. The most simple form of pattern recognition is a linear relation between x and y.
There is a wonderful website by Harvard statistician Tyler Vigen called Spurious Correlations, where he wonders and riffs about how variables work together. He gives all sorts of random and funny examples of things that correlate well but don’t cause each other - like what is the relationship between people who drown by falling in pools and Nicholas Cage films (66%!) or the divorce rate in Maine and consumption of margarine (turns out 99% correlation!)
Q: CAN YOU EXPLAIN EXACTLY WHAT YOU DO AND HOW YOU USE AI FOR WEATHER PREDICTION?
A: We are not using AI to directly predict the weather. What we are doing is using machine learning techniques to reduce errors in numerical weather forecasts. Weather is a chaotic system. Even with the best numerical models, there is a limit to how far out we can predict. We can say that the high temperature on this day will be 100, but that prediction skill loses specificity over time. We use models to predict not such a specific temperature over time but if there will be a warming trend over time.
We are using AI to discover how and where models are wrong and how we can correct that. We do this by analyzing a larger set of forecasts, and we look at how well those performed. From there, we can start to tease out when and where the model is wrong and make corrections. And when we understand errors in the forecast model, we can adjust to those.
Q: WHAT ARE SOME CURRENT AREAS WHERE AI IS USED FOR PREDICTION AND CORRELATION?
A: WeatherFlow-Tempest uses AI to correct errors in numerical weather predictions that are the basis of all human and digitally delivered forecasts. All numerical weather prediction models have errors. They might be general (winds always too strong by 10%), somewhat specific (nighttime lows too low by 5 degrees), or very specific (rain amounts in a city are twice what is observed during summer when winds are out of the North). It can take a human the better part of a lifetime to understand model errors for a single location. With enough data, AI allows us to discover and correct a significant number of errors in numerical weather prediction at hundreds of thousands of locations around the earth and continuously update that analysis throughout the day.
Q: CAN YOU GIVE ME AN EXAMPLE OF THIS?
A: So for storm prediction, most will look at radar. Radar imagery works well for the short term, but for the longer term, you need to model the data. One use case where it works is looking at the evolution of thunderstorms.
Q: WHAT ARE YOU DOING NOW?
A: Right now, we are correcting for standard atmospheric quantities - temperature, wind, humidity, and pressure. The next big ones are rain and lightning. Floods are definitely challenging, but better rain prediction using Tempest data would definitely be useful in flash flood forecasting because we are already able to model the flow of rain when it hits the earth's surface, so better rain prediction will lead to better flood warnings.
Q: HOW WOULD YOU DO THAT USING THE TEMPEST?
A: Radars don’t tell you the exact location of rainfall on the ground. Tempest measures the amount of rain with the haptic sensor on top of the Tempest weather station. The sensor is able to determine the size of raindrops. It works the same way a drum works - those signals are calibrated and translated into rainfall amounts. So with an array of our Tempest stations and other data, we can determine the location and rainfall amounts and accumulate that every minute throughout the day. This information helps us train up our AI-based rainfall models.
From right: At AMS 2023, WeatherFlow-Tempest CEO Buck Lyons, CTO David St. John, Chief Scientist Dr. Jeff Copeland, Senior Atmospheric Scientist Dr. Feng Gao, Atmospheric Scientist Dr. Stacey Kawecki, Meteorologist Brad Wells, Director of Software Development Corrine Butcher, Weather Data Scientist Dr. Brody Fuchs, and Chief Science Officer Dr. Martin Bell.
Q: CAN YOU EXPLAIN MACHINE LEARNING AND HOW THAT IS USED FOR WEATHER?
A: Machine learning takes the available data and discovers patterns in the datasets algorithmically. We can train the resulting AI models very quickly. We are also using it to look at past weather predictions to see what the models got right or wrong. The more data we have, the better the machine learning AI model will become. While the Tempest Weather System is still only a few years old as a product, and we only have a relatively short historical record from which we can train, the forecast improvements are remarkable
Q: WHAT CAN WE EXPECT AI TO DO IN THE FUTURE IN TERMS OF PREDICTION?
A: One of the big areas for research is how to incorporate soft or vague data from people, not sensors, in training AI models. Traditionally, data from sensors is used to train AI, a well-understood process. The challenge now is how to leverage performance feedback from individual users of predictions.
Q: WOULD YOU SAY THE BIGGEST BENEFIT OF AI AND ML IS TO IMPROVE THE FORECAST THAT TEMPEST USERS GET IN THEIR OWN BACKYARD?
A: Yes, we are training the models with observations coming from users’ Tempests sitting right in their own backyards. So their forecast is much more attuned to conditions they will experience on a day-to-day basis. But the benefit doesn’t stop there. Forecasts between Tempest stations are also dramatically improved by the AI improved forecasts at each Tempest location.
Q: HOW DID YOU GET INTO THIS WORK?
A: Over 25 years ago, I was working with an energy utility company in Australia. They wanted to improve their usage model because when it gets hot, people use the AC, and then there are brownouts and insufficient supply when demand spikes.
It was the first time I used neural network AI techniques and applied them to various locations around Sydney and Melbourne to predict high temps over the coming week. That was fun. It was a brand-new technique, and people said it was crazy to try it. No one had done it before. But what we predicted was a vast improvement over the status quo, similar to what we now see in our weather forecasting.
With AI, we could reduce the uncertainty of the forecast showing when it was going to be warm. And for an electrical utility, reducing that uncertainty allowed them to manage supply, keep customers happy, and saved them money by buying electricity ahead of time, ahead of peak pricing. What great business potential this showed, and that was only the beginning!
HOW TECHNOLOGY HAS IMPROVED FORECASTING
Forecasters in New York were eyeing a concerning storm system developing on the West Coast, sure it would become a significant snow event once it finally reached them a week later. As the system made its way east, it grew as predicted into one of the worst blizzards in New York history. As warnings of the deadly storm began to trickle out to the public days in advance of its arrival, some New York weather scientists began to reflect on the advances in forecasting technology compared to events like the Great Ice Storm of 1988, for which warnings came only hours in advance of the historic storm.
LAST YEAR WASN'T THE HOTTEST ON RECORD, BUT...
That doesn’t necessarily mean it’s time to celebrate. Similar to the year before, La Niña’s cooling effects kept the 2022 global temperatures out of all time record-breaking highs. But, much of the world still experienced more severe and/or frequent extreme weather events and their disastrous effects, intensified by the overall warming trajectory of the planet. La Niña is expected to continue weakening throughout 2023, with a possible re-emergence of an El Niño system later in the year, possibly setting up 2024 for the highest global temperatures on record.
AI PREDICTS 2 DEGREE WARMING BY MID-2030s
A study using artificial neural networks estimates that the planet could cross critical warming thresholds by the mid-2030s, even with the substantial reduction of pollutants and other emissions as proposed in the Paris Accords. The average global temperature has already climbed slightly over 1 degree since industrialization. Scientists have identified 1.5 degrees as a key tipping point for the climate, with a rise over 2 degrees potentially bringing devastating and irreversible effects. Using climate models and historical data on global temperatures, scientists could run three different prediction scenarios based on varying levels of emission reductions to reveal that global temperatures would hit 1.5 degrees of warming by 2035 in all scenarios.
WHY RECYCLING ISN'T THE ANSWER
We've heard it for years: Reduce, Reuse, Recycle. Recycling has always been a major part in our fight against plastic pollution, but research is showing that recycling can only get us so far to a sustainable plastic future. The real solution to transforming the plastic economy lies in making less in the first place.