On being a forecaster, on being a scientist

I was a scientist (if you include being a student) for about fourteen years, then a forecaster for seven years, and now a scientist for the last three. I sometimes wonder during which period I learned the most?

A forecaster learns about nature as it happens. The forecaster is sent in to study the situation, make an assessment and sometimes find out how it turned out. That person is confronted by real problems when things aren't working. He or she is able to see every pine needle on a few trees. Sometimes they are so close that sap sticks to their noses.

A scientist gets the broad perspective, studying things after the fact. I've done research involving thousands of catchments, looking at 30, 60, 90 years of data at a time. I can run experiments as if I was forecasting a long time ago, automating computer programs to do what I think my new techniques would have done back in 1995. Some scientists see the forest, the next valley over, off to the horizon. Birds fly below them.

There are other scientists that do field surveys, such as going out one summer and taking a lot of measurements in one place. That place isn't random, it might be in an instrumented pasture as a satellite flies overhead or it might be in the deepest snow around at the crest of spring.

Maybe it comes down to book smarts versus street smarts, education versus experience. Obviously both are important... it seems incomplete to have much more of one than the other. And you would hope that everyone would at least keep gaining either.

The first river forecaster I met

I'm sure the first river forecaster I met was Dallas Reigle, the Hydrologist for Salt River Project, the main water supplier for Phoenix, Arizona. I was a fresh faced grad student at University of Arizona in the hydrology department. He came in as a guest lecturer once a year as a special treat to the students and as a favor to the department. I remember sitting at cold stone-top benches in a darkened room as he showed the choicest clips of video from the 1993 floods. 

As much as it sounded like a character in a Dickens novel about Western Water, that really was his name. Dallas Glen Reigle... I never knew his middle name until I googled him- it makes him sound like a Gran Reserve limited batch of whiskey. Actually, Dallas Glen Reigle the Second (as if one wasn't enough). And he smoked a cigar a day. 

Arizona had been bopping along for a couple decades and then when the 1993 floods hit, the hydrologists were like "We're going to need a bigger chart!!!" It was unlike anything they'd seen before, major bridges getting washed away, reservoirs raging full blast. 

You must understand that this was in the days before Youtube. Now you can see anything you could ever want on the Web. Cats making funny faces, yeah the Internet has that. Dating sites for Ayn Rand fans, there's probably a couple to pick from. Back in the 1990s, however, the best you could do was order some tapes from your local television studio. 

But this was company footage he was showing. The hydrologists flew around the watershed in a helicopter, rapidly finding where the river was getting out of control. Can you think of the last time your work said to you, "We need your help, get to the helicopter!" Dallas peppered the video's narration with his own booming drawl that Westerners would call Southern and Southerners would call Western. At one point he said that a swing in the camera angle was because someone threw up in the helicopter as it swirled over the gushing spillways. Can you think of the last time someone threw up on you in a helicopter in the name of work (that didn't involve guns and missiles)? If you can, please message me. 

Roosevelt lake from the air. Biblical flood coming in from upper left. Note construction on right. Note water seeping through the face of the dam. Bad timing for all this to come together, I reckon. 

The video wasn't all disaster porn. 1993 was also the year that they had been finishing years of improvements to the main reservoir. Lots of things still under construction got ruined. We learned about cofferdams and other technical details... What went wrong, what went well, etc. It was a classroom in an Engineering college after all. 

His sense of humor was purely and infinitely dry. He only laughed at inappropriate times. In the middle of his slides he showed a photo of a clown with a frown face. It wasn't a happy clown with a frown, it was a run-down dirty clown that looked disoriented. Dallas said "What was that? Who put that in there? gah. Next slide please".... Years later I tried the same trick with a group of 2nd graders. It had about the opposite effect that I intended, anarchy broke out and it didn't settle down until I left.  

There in that classroom, I was pie-eyed. That was it, there was no turning back, I wanted to be that guy. 

The best forecast

My interest in forecast evaluation (saying how good forecasts are) goes back to about 1998 when a ginormous El Nino was threatening California and Arizona with floods. My Masters thesis was on how water providers and emergency managers used those forecasts from September to say what might happen that coming winter.

I did long interviews with key people in Arizona. I went some strange places, particularly emergency management offices. I saw the "big board" at the state emergency center. They really do have black helicopters at the Phoenix bunker (yes, a bunker with zigzagged hallways set up for nuclear explosions).

Before the start of the winter, a late season Pacific hurricane came up the west cost and passed through Yuma and California/Arizona border. That hurricane, and the images of the raging floods during the most-recent-ginormous-El-Nino in 1983, were enough to put the fear of god in everybody. Seriously, I'd go into flood managers' offices and at reception they'd have a massive photo of roiling waves and houses washing away from 1983- you would think that would be fresh in their mind. It wasn't this picture, but this was the event:

Anyhow, you didn't want to be that guy that everybody warned but you didn't do anything and then it happened and jeepers, what do we even pay you for anyway?!?

After the event, many people were wondering, there's small and medium El Ninos going on all the time, is this kind of warning something we could use all the time? How good are the forecasts? Will they ever bite us?

Well, it turns out, to a user, "how good are they" is a very complicated question that depends on where you are, what you do, how much risk you can handle and a host of other things. But for a forecaster, interestingly, there's only a few ways to be right and a few things to strive for to be good.

In my opinion, the best writer on "the goodness of forecasts" was Allan Murphy. I never met him, but in grad school my copy of his collected works was dog-eared and tattered. I hope to weave some of his ideas into my work this year.

Tom

Obama on instincts

This from a recent interview with Obama on 60 minutes

"The thing about gut instinct is if it works, then you think, "Boy, I had good instincts." If it doesn't, then you're gonna be running back in your mind all the things that told you maybe you shouldn't have done it."

I waffle around a lot about the value of instincts. In my mind, I always have line of "Yes, but..."s ready to go on any decision. Looking back, any or all of them could have been right. But history is written by the victors, so I forget the wrong instincts and remember the right one. Instincts are always plural, a chorus of notes, one or two bound to be in-tune.

Probably one of the biggest ways my personal philosophy changed (by being a forecaster) was by realizing that "I am not lucky." Not good lucky, not bad lucky. There's chance, of course... but not luck. I'm no more or less likely than anyone else to win at the casino. I'd like to buy low and sell high, but my hopes/dreams/fears will have nothing to do with the outcome.

It sounds cold, I know, but it's a liberating realization.

Shopping for interviewing equipment

Over the next year I'm hoping to do a good number of interviews. A few folks suggested getting a handheld recorder, rather than having to worry about taking good notes. I sort of think that a tape recorder is less intimidating than a full camera. Despite any practicing or being told to relax, it is difficult getting comfortable at the end of the barrel of a camera.

I'm always baffled to see that something selling for $140 on the web in the US is $210 in Australia. Even the $30 international shipping doesn't make up the difference. Neither do currency differences- you could change 3 Aus dollars into 2 US dollars a couple years ago but now the AU$ is worth more than the US$. Someone once suggested that you can get great deals by searching for something misspelled ("camerra"or "camra" instead of "camera"). I can think of some great automated arbitrage that could happen here.

Do you know much about audio equipment and/or interview techniques? I'm all ears!

Water, like religion

Also saw this at the zoo

"Water, like religion and ideology, has the power to move millions of people. Since the very birth of human civilization, people have moved to settle close to it. People move when there is too little of it. People move when there is too much of it. People journey down it. People write, sing and dance about it. People fight over it. And all people, everywhere and every day, need it.We need it for drinking, for cooking, for washing, for food, for industry, for energy, for transport, for rituals, for fun, for life. And it is not only we humans who need it; all life is dependent on water to survive."

Mikhail Gorbachev

The problems of water

Seen on a sign at the Werribee Zoo

Anyone who can solve the problems of water will be worthy of two Nobel prizes- one for peace and one for science. - John F Kennedy

On being wrong

When forecasting, no one is ever exactly right. Instead, there are varying degrees and flavors of being wrong. Nobody likes to be wrong.

Kathryn Schulz has a great talk online titled "On being wrong" (linked below). At one point she asks a question of a few people in the front row:

"So let me ask you guys something ... How does it feel -- emotionally -- how does it feel to be wrong? [Audience responds] Dreadful. Thumbs down. Embarrassing.... thank you, these are great answers, but they're answers to a different question. You guys are answering the question: How does it feel to realize you're wrong? [Audience laughter] Realizing you're wrong can feel like all of that and a lot of other things, right? I mean it can be devastating, it can be revelatory, it can actually be quite funny.... But just being wrong doesn't feel like anything."

Forecasting requires a certain form of mental training that involves a strong sense of self awareness. Are you fooling yourself about what you think might happen? Is your intuition helping or hurting you? Are you justified in being self-confident? Or does it all come crashing down when you're surprised by the outcome?

Ultimately, one learns to make no-regrets decisions based on evidence and a dash of gut-feeling. It is a skill that I feel would help most people navigate through uncertainty in their daily lives. I'm hoping to explore this more over the coming year. Enjoy the video!

Cold Missouri Waters

River forecasters have a good deal in common with forecasters of other natural disasters, such as wildfires (called bushfires here in Australia). They inform response teams in the field that try to protect lives and property. Snap decisions have to be made under pressure. Lastly, you can always be sure that this coming year is going to be the weirdest yet.

Folk band Cry Cry Cry did a cover of "Cold Missouri Waters", a true story of a fire that seemed controllable but eventually consumed a number of fire fighters. Dodge is the main character and is speaking from his deathbed (Hodgkin's disease) 5 years after the fire. He is haunted because he was responsible for thirteen people who died. Forecasters, like firefighters have to deal with many ordinary, even boring situations. But when action happens, it comes quickly. Here are the lyrics.

My name is Dodge, but then you know that

It's written on the chart there at the foot end of the bed

They think I'm blind, I can't read it

I've read it every word, and every word it says is death

So, Confession - is that the reason that you came

Get it off my chest before I check out of the game

Since you mention it, well there's thirteen things I'll name

Thirteen crosses high above the cold Missouri waters

August 'Forty-Nine, north Montana

The hottest day on record, the forest tinder dry

Lightning strikes in the mountains

I was crew chief at the jump base, I prepared the boys to fly

Pick the drop zone, C-47 comes in low

Feel the tap upon your leg that tells you go

See the circle of the fire down below

Fifteen of us dropped above the cold Missouri waters

Gauged the fire, I'd seen bigger

So I ordered them to sidehill and we'd fight it from below

We'd have our backs to the river

We'd have it licked by morning even if we took it slow

But the fire crowned, jumped the valley just ahead

There was no way down, headed for the ridge instead

Too big to fight it, we'd have to fight that slope instead

Flames one step behind above the cold Missouri waters

Sky had turned red, smoke was boiling

Two hundred yards to safety, death was fifty yards behind

I don't know why I just thought it

I struck a match to waist high grass running out of time

Tried to tell them, Step into this fire I set

We can't make it, this is the only chance you'll get

But they cursed me, ran for the rocks above instead

I lay face down and prayed above the cold Missouri waters

And when I rose, like the phoenix

In that world reduced to ashes there were none but two survived

I stayed that night and one day after

Carried bodies to the river, wonder how I stayed alive

Thirteen stations of the cross to mark to their fall

I've had my say, I'll confess to nothing more

I'll join them now, those that left me long before

Thirteen crosses high above the cold Missouri waters

Thirteen crosses high above the cold Missouri shore

The party's over even before it got started

Wilson's Prom pictures from Parks Victoria.

As the time for our year of travel comes closer, we're trying to tick all the boxes on things we'd like to do before we leave. If you say you want to do overnight hikes in nature near Melbourne, the Great Ocean Walk is usually the first thing people say... "But if you can handle a bit more driving you have to go to Wilson's Promontory."

On the tip of "The Prom" is a lighthouse that has cabins with lots of bunk beds and shared kitchens, just bring your food and sleeping bag... maybe bring your life size body pillow if you can't live without it. We got reservations months in advance and Easter is one of the craziest times of year for tourists.

Well, it turns out that two years ago the Prom was burned badly by wild fires (bush fires). And then a couple weeks ago massive flooding hit and destroyed a lot of infrastructure.

Apparently it was like someone took the prom, tore it up in little pieces and threw it up in the air. Nobody is getting in there anytime soon. Our reservation is not going to help us get across that gap in the bridge.

Plan B may be wreck scuba diving the Ex-HMS Canberra.

Inundated with goats

Here's a headline you don't see too often...

Roma to be inundated with goats

The floodwater has cleared from the streets of the southern Queensland township in Roma just in time for the annual goat racing carnival. Roma had its fourth major flood in just over a year when the water peaked at 7.6 metres earlier in the week...."Goats will run up the road all right, but put a cart in behind them and they'll want to roll over or go backwards and do all the things that shouldn't be done," he said.

Below is a chart from the Bureau of Meteorology webpage for a nearby river:

Bungil Ck at Tindarra

It may be a bit hard to see, but each vertical line is a day and this shows the last 8 days. The higher the line is, the deeper the river is. The three horizontal words are the flood levels of minor, moderate and major (which roughly translates into "that's annoying", "hey, that's not good" and "sweet lord, this is the end!")

For days and days it was flat at zero, a totally dry channel. Then in 20 minutes there was 4 feet (1.5m) of water, in 2 hours it was 10 feet (3.5m) and in 12 hours it was 24 feet (8m) deep. Even then, they still had enough advance warning to sandbag the town.

All the more remarkable is how quickly it has dropped. Now it's back to only a couple feet deep 2 days later, just in time for Easter weekend.

To me, the graph looks like an elephant inside a snake. Maybe it's a goat?

Chinese weather satellites

This too was a post from a former blog. This is about the Chinese weather remote sensing center.

This picture was taken at the Chinese Meteorological Agency in Beijing in November 2005. I went there for a workshop on monitoring drought. They wanted to set up a drought monitor like we have in the US. It was a mix of talks and table-top type exercises. The food was amazing... Our hosts taught us the saying that the Chinese eat everything with four legs except the table.

This picture was a tour of their central command for managing all the weather satellites (i.e. location, speed, how all the sensors are doing). The man in black I think is the head of the Chinese Academy of Science's environmental division? If you looked to our right there was another bank of computers and then a large display on the wall. I vaguely remember that it wasn't a projection, it was a 5x15 foot LCD screen. I went back to China in 2010 and saw a few more of these wall-sized LCD monitors in meeting rooms.

One strange thing you'll notice is what they're wearing. Whenever someone was doing anything operational (e.g. making a forecast, preparing a map) they'd have on a white lab coat. Not many chemicals to protect yourself from in weather monitoring, but it definitely gave that air of scientist-as-the-modern-techno-priest, separate from the laymen.

During my visit, I found the Chinese Meteorological Agency was surprisingly innovative; it was making products that I had not heard of in the US or Australia. They also had very long records of the weather.

Queensland flood forecasters

This is a post from a former blog of mine in March 2010. It describes a visit to the Bureau of Meteorology's flood warning office in Brisbane. I hope to be doing more visits like this in the coming year.

I was in Brisbane because Queensland had recently been in floods and it was anticipating that floods might be coming up. I do research on how to make better short-term (less than 10 day ahead) streamflow forecasts. So the Bureau of Meteorology invited me up to see the flood warning office in full swing.

Here's their building on at 69 Ann St:

It's a professional and modern building, with stunning views from the 21st floor. The flood warning center shares office space with the weather forecasters. Here's one of the meteorologists at work.

I had monitor envy. The guy who does data quality control and monitoring of the rainfall systems gets to work at a station with 6 screens:

Inland Queensland had been flooding on and off since Christmas. During the visit there was a weather briefing from the meteorologists, saying that there was near uniform agreement from the weather models that a major storm was going to cross down from the north coast into the interior. About 5 days ahead, the forecast was that it would drop more than 100 mm in a day, which is about 4 inches.

In addition to output from models run on some of the world's most powerful computers, there's still a lot of work that happens drawing weather patterns on paper maps with colored pencils:

Yearly average rainfall for the interior is about 200-300 mm. So, yeah, 100 mm's a huge deal, especially since the ground was already saturated!

On the left is Peter Baddiley, the head of the Queensland flood forecasting group. When the rainfall scenarios were run through the river models, the response was epic floods. However, 5 days ahead there's so much uncertainty, you try and be very cautious to avoid false alarms. Although a general alert wasn't sent out that day, high level government authorities (including an emergency management representative who had a jacket with the agency's acronym on the back) were brought in and briefed on the situation while I was there.

It brought me right back to when I was an operational forecasting in the US. Is it going to happen, is it not going to happen? Can you trust your models? Could it be even worse than the models suggest? The thrill, the drama, the anticipation. At the same time, you have to not let your emotions get in the way; Pardon the gruesome imagery, but it reminds me of this quote from the movie Jaws

"Sometimes that shark he looks right into ya. Right into your eyes. And, you know, the thing about a shark... he's got lifeless eyes. Black eyes. Like a doll's eyes. When he comes at ya, doesn't seem to be living... until he bites ya, and those black eyes roll over white and then... ah then you hear that terrible high-pitched screamin'."

Nature doesn't care about you. Doesn't care what you think is going to happen. You may fear this or that, or maybe last night you saw a movie about something that plants the seed in your mind of a big drought or flood. But the important thing is to be level headed and objective, pay attention to what the evidence is telling you and always be aware of self-delusion.

In the end, the rainfall was ginormous, widespread areas of heavy rainfall.

The flood response has been big too, large areas in the Major Flood category.

It reminded me that the favorite part of my last job as an operational forecaster was being in that situation of knowing that the tools I had at the time were functional but could be improved. I'd go off and do a bit of tinkering and development and then next time around we'd be better armed to know what was really going to happen. When the event finally unfolded like we thought it would, there was a mix of disbelief and ecstasy that it actually worked and we had been able to discover Truth before it happened. So rarely in life do you discover with certainty that your belief is right or wrong, often you can get by with just arguing for a convincing position. I'd like to think having that experience built character.

My father loved gambling but my mother hated it, so at the very least I got to satisfy that betting urge without losing any money!

If only we knew

When I was growing up, I would always fantasize about time travel and what I could do if I knew the future. Usually I would want to be a fly on the wall at some historic event, or assassinate a dictator... but as Reaganomics came along, it was explained to me that I should play every move of the stock market to become infinitely rich.

Of course all this is complicated by, say, the paradox caused by meeting myself or changing the course of history. However, nature doesn't know how much people know about it, nor does it care. So it would be OK for the time traveller (above) to go and find out the weather, right?

When forecasting, sometimes the suspense of not knowing was almost too much to bear. "Just once" I thought "I want to know how this will all turn out". I wanted to cheat, to know what nature's cards were, to know how to bet. Perhaps I'd have to sell my soul in an infernal pact but would it be worth it?

The worst part of it all is that so many people are watching rivers and the weather in realtime, it's hard to know who knows what they're talking about. Even if I knew the answer I couldn't explain why I knew... nobody would believe me. I'd say what will happen, it'll happen and they'll think it was a lucky guess.

I met many people who were confident in themselves but, again, nature doesn't care and humiliation is never far off. Seems like knowing the future for sure would be more of a curse than a blessing.

If you could go to the future and know one thing, what would it be? And how would you convince others that you had the right answer?

Whitsett Intake Pumping Plant

In grad school for Hydrology from University of Arizona I once took a 1 credit field trip down the Colorado River. We started at Hoover Dam and drove down to Mexico, talking with water managers and users all down the way. We took water quality samples (by the time we reached Mexico the water being pumped stank of rotten eggs). We asked everyone "who runs the Colorado?" and everyone had a different answer (usually "I do").

One place I remember was a Californian pumping station pictured above (not my photos, look here).

The water comes out of the river and up the pumps through the mountain up to lakes and more pump stations for about 250 miles (400 km).

This place, built in the 1930s has art deco touches. They keep it clean as a cathedral. I went to many churches when growing up so the feeling of being there was familiar.

The quality of construction is beyond anything you would see today, almost like science fiction, but in reverse. The future already happened during the Grand Times of the 1930s and is gone now except for some of the buildings.

In the basement is this turbine/pump. The inner silver metal piece is spinning at a blur. With the consent of our hosts, I reached over the railing and balanced a 5 cent piece on its side on the green metal near the base. This thing is perfectly balanced and has no vibration at all. I took a picture of the nickel while it was standing on its own but lost it when I had a hard-drive crash years ago.

This year I want to visit some amazing structures, behind the scenes. It is getting very difficult with increased security. I have never met anyone who has been inside Hoover Dam's control room, not even Bureau of Reclamation employees.

If you know a place I can go, please let me know! Tom

Welcome!

Everyday around the world, small groups of river forecasters study the state of nature, put science to the test, and have the results held out in front of them. There's no cheating on this exam and there is no denying the score.

Some of the most horrific disasters happen when water gets out of control. While floods come to mind, droughts do more damage with their slow chaos. Yet, when disasters are not happening, society goes on with the daily use of water to keep civilization humming. The forecaster is there all the time, in good times and in bad.

When nature signals its intentions, the forecaster should be the first to know. He or she would then need to know who to contact next. The message should be clear, specific and relevant, able to affect a decision.

I love river forecasting. There, I said it. It feels good.

Do you love forecasting too? Or want to know some more? Leave a comment!

Tom Pagano