From models to forecasts and what humans add in-between (part 1)

The recent situation in Manila reminded me of a dilemma I had when I was a forecaster, one that strikes at the heart of how humans can both help and hurt the forecasts. This post talks about some of what humans have to consider when making river forecasts. Before I jump in to this, I will add the caveat that I do not know what PAGASA forecasters in the Philippines felt about their situation over this last month- this is just based on my experience in the US and what my thought process was in similar cases.

To recap for Manila, two years ago catastrophic floods resulted from back-to-back typhoons, one over Manila and another some distance to its north. Then, last year, a typhoon was forecast to miss Manila, passing to its north, but instead resulted in a direct hit to the city. It wasn’t a terrible flood, but the surprise left many unhappy. A few months later the head of the weather service was replaced, after many years of service.

A statue was built along the waterfront to commemorate the two year anniversary of the flood disaster that killed hundreds. By an incredible coincidence, the statue was under water at its official unveiling because a major typhoon had just passed through Manila again, this time killing 83.   

High water at Marikina in Manila from Typhoon Nesat 

The flood disaster statue named Marikitna. For scale, see a human on the lower right. 

How the statue looked the day after its unveiling (source)

When I visited it after the flood the statue had debris half way up it, attached to the rope. The base also had large cracks (lower left)

High water debris attached to a ladder (see arrow) near Sto. Nino gaging station in Marikina. Human in background for scale. 

You could guess that the soils were nearly saturated from the recent high water. Then, almost as soon as the first Typhoon was finished, another was forecast to pass near Manila, but likely more to the north than through the city itself. 

What would I be thinking if I was the forecaster in that situation? For one, I would be under enormous pressure and stress, surging with adrenaline, working long hours. The mood in the office would be something like the lyrics from the song “99 Red Balloons”:

The hit from the 1980s about false alarms (source)

Panic lads, it’s a red alert

There’s something here from somewhere else …

Ninety-nine ministers meet

To worry, worry, super-scurry

Call the troops out in a hurry

This is what we’ve waited for

This is it boys, this is war

The President is on the line…

That song is about trying to figure out if a detected nuclear missile launch is real or not and if retaliation is necessary. In my case as a river forecaster, I would similarly be wondering if there was a threat from the second typhoon. I would have a sense that this was a big, big forecast, if there was ever a need to get one forecast right, this would be it. Every forecast is important, but these would be highly visible forecasts and it would be my chance to help many people… or mess up many people, most of whom I’d never meet.

So the first question would be, “what do the model(s) say is going to happen?” Models come in many forms, but basically they are quantitative summaries of how people believe nature works. They are the vessels that scientists pour their knowledge in to. Some models are built by piecing together our understanding of how individual processes work (e.g. one person may study how much plants evaporate when the soil is dry… or how quickly water travels through sandy soils).

Some models need more data than others; to set up some models, for example, you might have to know much sand is in your area and where? You might need long records of rainfall and river data. Has someone been diligently measuring rainfall in one location for years and years? And can he tell you quickly how much rain fell in the last hour? If you’re a lucky forecaster, you have lots of access to data. Most agencies are not lucky.

Furthermore, models are not things that you gin up easily or quickly. If forecasting was battle and models were weapons, when the enemy comes charging over the hill, you can’t expect to grab the nearest rocks and vines and turn them into an assault weapon. That said, many operational forecasting services only have the hydrological modelling equivalent of pointy sticks. As Rumsfeld says “you go to war with the army you have---not the army you might want or wish to have at a later time” As far as I could tell when I was there, the Manila hydrologists were not running any computer hydrologic models. We used hydrologic models in the US but they were old compared to the state of the art in the research community. I suspect that Thailand’s models are on par with those used in the US.

Not to stretch the war metaphor too far, but relatively vintage hydrologic models are widely used for much the same reasons that the AK-47 rifle remains the world’s weapon of choice, even though it was invented 60 years ago. These classic standards are good enough for most situations and don’t require anything fancy for maintenance and operation.

Both the M-16 and AK-47 were invented decades ago although they are still popular, much like hydrologic models from the 1970s (source)

So the model would gonkulate away and spit out some numbers about what the river might do. The next question is “Is there anything I know about in nature that is not included in the models that is going to be important?” In the Manila forecasting office were a set of charts that showed “If this reservoir is X% full and Y millimetres of rain falls, then the final level of the reservoir will be Z%”. The charts are handy because they could be used for other “what if?” questions, e.g. “if the reservoir is X% full, how much rain would be needed to make the reservoir 100% full?” The math of it all is very simple- 100 millimetres of rain over the 100 square kilometres of catchment would add 10 million cubic meters of water to the reservoir.

That assumes all the rain becomes runoff but often that’s not the case. Let’s just say half the rain becomes runoff (“50% runoff efficiency”) and the other half goes elsewhere, maybe as evaporation or recharge down to deeper groundwater. So 100 millimetres of rain on the land becomes 50 millimetres of runoff in the stream and that would add 5 million cubic meters to the reservoir. But what if it’s not half? What if it’s 60%? Or 80%? In the Manila office there was a thick folder where each page had a chart for different levels of runoff efficiency, 40%, 50%, 60%, 70%, 80% and so on. If I was a forecaster and that was my tool, it would be my job to try and get on the same page as nature, literally and figuratively.

Reading off "nomograms" (lower left) at PAGASA's flood center in Manila to determine how much rainfall would be needed to fill the reservoirs. 

An example "nomogram" relating how full the reservoir is at the beginning, how much rainfall is expected and how full the reservoir will be at the end. For example, the reservoir starts at 50 feet deep ("A") and we expect 2 inches of rain in the future ("B"). Find where those two intersect and read off the level of the sloped line (in this case blue, 75 feet full). This assumes that half the rainfall becomes runoff. 

The same chart as above but this time assuming that 90% of the rainfall becomes streamflow. Now 2 inches of rainfall would leave the reservoir completely full (the two dashed lines meet at the black line). This is a much more serious situation. It is the forecaster's job to figure out if the runoff efficiency is, for example, going to be closer to 50% or 90%.   

How would I know which chart to use? For one, I could go back and look at other floods in this same place and see what happened in the past. Maybe I would have data from 10 flood events and they all hovered around the 50-70% runoff efficiency range. I’d ask what the floods at the extreme ends had in common. Perhaps the low efficiency events happened at the start of the typhoon season and the high efficiency events happened at the end. The highest efficiency ones were when two typhoons struck back to back. That makes sense- at the beginning of the season the soils are dry and absorb more of the rain and at the end they’re already primed and the rain goes directly to the streams. 

There can be information not included in the model, but that may also not be relevant. Let’s say the two biggest floods happened during full moons. Is there a connection? It could just be coincidence. Full moons are associated with high tides so maybe the water is backing up into the city and making the floods worse. I, as a human, could make the judgement call that full moons are irrelevant because we’re too far from the ocean.

But wouldn’t I feel like a real idiot if the next big flood happens during another full moon? I had the information but decided to ignore it. Nature always sides with the hidden flaw and the flaw never stays hidden for long. It would be just my luck to have that hairline crack in my concept of nature break wide open at the next forecast. The problem is that there are so many patterns (so many hairline cracks) that are not reliable enough to be useful (big enough to worry about). Superstitiousness is not a positive trait among forecasters. But the incentive to get the forecast right is so incredibly high, I would always be mentally filing away bits of information in my mind’s junk drawer.


(Continued in the next post...)