mon, 24-dec-2012, 07:37

Thinking a little more about my previous post on the cold winter we’ve been experiencing for the past three months, it would be easier to understand the analysis if I reported average temperature for the period, rather than a cumulative value. I also realized that if there were missing days in any of the record, that this would skew the results against those years with missing data. So here’s a better query and the results through December 23rd:

```SELECT year, round(cum_f / n, 2) AS avg_tempf,
rank() OVER (ORDER BY cum_f / n) AS rank
FROM (
SELECT year,
CASE WHEN year = 2012 THEN round(sum(tavg_f), 2) - 261.5
ELSE round(sum(tavg_f), 2)
END AS cum_f,
CASE WHEN year = 2012 THEN count(*) + 7
ELSE count(*)
END AS n
FROM (
SELECT extract(year from dte) AS year,
extract(doy from dte) AS doy,
(tmin_f + tmax_f) / 2.0 AS tavg_f
FROM get_ghcnd('Fairbanks Intl Ap')
ORDER BY year, doy
) AS foo
WHERE doy between extract(doy from '2011-10-01'::date)
AND extract(doy from '2012-12-23'::date)
GROUP BY year
ORDER BY year
) AS bar
ORDER BY rank;
```
Average, minimum, maximum temperature (°F) for October 1st through December 23rd, ranked by average temperature over the period.
Rank Year Average temp (°F) Minimum temp (°F) Maximum temp (°F)
1 1917 -4.11 -52 57
2 1956 -3.99 -50 46
3 2012 -0.74 -47 61
4 1975 -0.01 -51 57
5 1977 0.54 -52 53
6 1942 0.92 -50 63
7 1946 0.95 -57 55
8 1961 1.33 -54 53
9 1996 1.75 -36 41
10 1966 1.92 -53 61

We’ve moved to third place on the list, with an average temperature over the period of less than zero Fahrenheit.

Here’s the same analysis for November and December (through the 23rd):

Average, minimum, maximum temperature (°F) for November 1st through December 23rd, ranked by average temperature over the period.
Rank Year Average temp (°F) Minimum temp (°F) Maximum temp (°F)
1 1917 -20.46 -52 21
2 1956 -16.81 -50 44
3 1946 -16.34 -57 33
4 1942 -14.63 -50 21
5 2012 -14.34 -47 31
6 1977 -13.86 -52 21
7 1975 -13.71 -51 30
8 1966 -10.91 -53 32
9 1906 -10.72 -50 37
10 1961 -10.05 -54 42

The Weather Service is forecasting some warmer weather starting today, so it’s likely we’ll begin to drop down the list, but it will be interesting to see where we wind up at the end of December.

Regardless, it’s been an exceptionally cold October, November and December in Fairbanks. We’re all just hanging on, hoping our houses, vehicles and other equipment continues to function until we get some warmer weather.

fri, 21-dec-2012, 05:52

The 2013 Tournament of Books book list came out yesterday: ToB selections

Here they are, with three star rating system for those I've read, ☆ - not worthy, ☆☆ - good, ★★★ - great. I couldn’t finish Ivyland and wasn’t interested in Dear Life at all, so neither of those are contenders for me.

Main tournament:

Pre-Tournament Playoff Round:

I’ve now read all the main tournament books I’m going to, and two of the “Iraq / Afghanistan” books that are in the Pre-Tournament Playoff. My current pick is Billy Lynn’s Long Halftime Walk, but I wouldn’t be upset if Building Stories, The Fault in Our Stars, Gone Girl, HHhH or Bring Up the Bodies wound up winning. The Orphan Master’s Son is also excellent, but maybe a shade below the other five. Still left to read is the other pre-tournament pick (and the two others I don’t want to waste my time on…).

Updated 2013-01-28

thu, 20-dec-2012, 19:16

-45.8°F outside

It seems like it’s been cold for almost the entire winter this year, with the exception of the few days last week when we got more than 16 inches of snow. Unfortunately, it’s been hard to enjoy it, with daily high temperatures typically well below -20°F.

Let’s see how this winter ranks among the early-season Fairbanks winters going back to 1904. To get an estimate of how cold the winter is, I’m adding together all the daily average temperatures (in degrees Celsius) for each day from October 1st through yesterday. Lower values for this sum indicate colder winters.

Here’s the SQL query. The CASE WHEN stuff is to include the recent data that isn’t in the database I was querying.

```SELECT year,
CASE WHEN year = 2012 THEN cum_deg - 112 ELSE cum_deg END AS cum_deg,
rank() OVER (
ORDER BY CASE WHEN year = 2012 THEN cum_deg - 112 ELSE cum_deg END
) AS rank
FROM (
SELECT year, round(sum(tavg_c) AS cum_deg
FROM (
SELECT extract(year from dte) AS year,
extract(doy from dte) AS doy,
tavg_c
FROM ghcnd_obs
WHERE station_id = 'USW00026411'
ORDER BY year, doy
) AS foo
WHERE doy between extract(doy from '2011-10-01'::date)
and extract(doy from '2012-12-19'::date)
GROUP BY year ORDER BY year
) AS bar
ORDER by rank;
```

And the results: this has been the fifth coldest early-season winter since 1904.

Rank of cumulative degrees since 1904, thru December 19th
O/N/D rank year O/N/D cumulative °C N/D rank
1 1917 -1550 1
2 1956 -1545 4
3 1977 -1451 3
4 1975 -1444 5
5 2012 -1388 7
6 1946 -1380 2
7 1999 -1337 12
8 1966 -1305 9
9 1942 -1303 6
10 1935 -1298 10

In addition to the ranks for October through today (O/N/D rank in the table), the last column (N/D rank) shows the same calculation without October temperatures. It’s always a good idea to examine how well a relationship holds up when the interval is manipulated in order to see if the results are an artifact of the choice of period. In this case, the rankings change, but not dramatically.

Tonight we may cross -50°F for the first time this year at our house, but even without the very cold temperatures predicted (but not record cold) through the weekend, the start of the 2012/2013 winter has been exceptionally chilly.

tags: SQL  weather  winter  cold
sat, 01-dec-2012, 07:41

It’s now December 1st and the last time we got new snow was on November 11th. In my last post I looked at the lengths of snow-free periods in the available weather data for Fairbanks, now at 20 days. That’s a long time, but what I’m interested in looking at today is whether the monthly pattern of snowfall in Fairbanks is changing.

The Alaska Dog Musher’s Association holds a series of weekly sprint races starting at the beginning of December. For the past several years—and this year—there hasn’t been enough snow to hold the earliest of the races because it takes a certain depth of snowpack to allow a snow hook to hold a team back should the driver need to stop. I’m curious to know if scheduling a bunch of races in December and early January is wishful thinking, or if we used to get a lot of snow earlier in the season than we do now. In other words, has the pattern of snowfall in Fairbanks changed?

One way to get at this is to look at the earliest data in the “winter year” (which I’m defining as starting on September 1st, since we do sometimes get significant snowfall in September) when 12 inches of snow has fallen. Here’s what that relationship looks like:

And the results from a linear regression:

```Call:
lm(formula = winter_doy ~ winter_year, data = first_foot)

Residuals:
Min      1Q  Median      3Q     Max
-60.676 -25.149  -0.596  20.984  77.152

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -498.5005   462.7571  -1.077    0.286
winter_year    0.3067     0.2336   1.313    0.194

Residual standard error: 33.81 on 60 degrees of freedom
Multiple R-squared: 0.02793,    Adjusted R-squared: 0.01173
F-statistic: 1.724 on 1 and 60 DF,  p-value: 0.1942
```

According to these results the date of the first foot of snow is getting later in the year, but it’s not significant, so we can’t say with any authority that the pattern we see isn’t just random. Worse, this analysis could be confounded by what appears to be a decline in the total yearly snowfall in Fairbanks:

This relationship (less snow every year) has even less statistical significance. If we combine the two analyses, however, there is a significant relationship:

```Call:
lm(formula = winter_year ~ winter_doy * snow, data = yearly_data)

Residuals:
Min     1Q Median     3Q    Max
-35.15 -11.78   0.49  14.15  32.13

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)      1.947e+03  2.082e+01  93.520   <2e-16 ***
winter_doy       4.297e-01  1.869e-01   2.299   0.0251 *
snow             5.248e-01  2.877e-01   1.824   0.0733 .
winter_doy:snow -7.022e-03  3.184e-03  -2.206   0.0314 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 17.95 on 58 degrees of freedom
Multiple R-squared: 0.1078,     Adjusted R-squared: 0.06163
F-statistic: 2.336 on 3 and 58 DF,  p-value: 0.08317
```

Here we’re “predicting” winter year based on the yearly snowfall, the first date where a foot of snow had fallen, and the interaction between the two. Despite the near-significance of the model and the parameters, it doesn’t do a very good job of explaining the data (almost 90% of the variation is unexplained by this model).

One problem with boiling the data down into a single (or two) values for each year is that we’re reducing the amount of data being analyzed, lowering our power to detect a significant relationship between the pattern of snowfall and year. Here’s what the overall pattern for all years looks like:

And the individual plots for each year in the record:

Because “winter month” isn’t a continuous variable, we can’t use normal linear regression to evaluate the relationship between year and monthly snowfall. Instead we’ll use multinominal logistic regression to investigate the relationship between which month is the snowiest, and year:

```library(nnet)
model <- multinom(data = snowiest_month, winter_month ~ winter_year)
summary(model)

Call:
multinom(formula = winter_month ~ winter_year, data = snowiest_month)

Coefficients:
(Intercept)  winter_year
3    30.66572 -0.015149192
4    62.88013 -0.031771508
5    38.97096 -0.019623059
6    13.66039 -0.006941225
7   -68.88398  0.034023510
8   -79.64274  0.039217108

Std. Errors:
(Intercept)  winter_year
3 9.992962e-08 0.0001979617
4 1.158940e-07 0.0002289479
5 1.120780e-07 0.0002218092
6 1.170249e-07 0.0002320081
7 1.668613e-07 0.0003326432
8 1.955969e-07 0.0003901701

Residual Deviance: 221.5413
AIC: 245.5413
```

I’m not exactly sure how to interpret the results, but typically you’re looking to see if the intercepts and coefficients are significantly different from zero. If you look at the difference in magnitude between the coefficients and the standard errors, it appears they are significantly different from zero, which would imply they are statistically significant.

In order to examine what they have to say, we’ll calculate the probability curves for whether each month will wind up as the snowiest month, and plot the results by year.

```fit_snowiest <- data.frame(winter_year = 1949:2012)
probs <- cbind(fit_snowiest, predict(model, newdata = fit_snowiest, "probs"))
probs.melted <- melt(probs, id.vars = 'winter_year')
names(probs.melted) <- c('winter_year', 'winter_month', 'probability')
probs.melted\$month <- factor(probs.melted\$winter_month)
levels(probs.melted\$month) <- \
list('oct' = 2, 'nov' = 3, 'dec' = 4, 'jan' = 5, 'feb' = 6, 'mar' = 7, 'apr' = 8)
q <- ggplot(data = probs.melted, aes(x = winter_year, y = probability, colour = month))
q + theme_bw() + geom_line(size = 1) + scale_y_continuous(name = "Model probability") \
+ scale_x_continuous(name = 'Winter year', breaks = seq(1945, 2015, 5)) \
+ ggtitle('Snowiest month probabilities by year from logistic regression model,\n
Fairbanks Airport station') \
+ scale_colour_manual(values = \
c("violet", "blue", "cyan", "green", "#FFCC00", "orange", "red"))
```

The result:

Here’s how you interpret this graph. Each line shows how likely it is that a month will be the snowiest month (November is always the snowiest month because it always has the highest probabilities). The order of the lines for any year indicates the monthly order of snowiness (in 1950, November, December and January were predicted to be the snowiest months, in that order), and months with a negative slope are getting less snowy overall (November, December, January).

November is the snowiest month for all years, but it’s declining, as is snow in December and January. October, February, March and April are increasing. From these results, it appears that we’re getting more snow at the very beginning (October) and at the end of the winter, and less in the middle of the winter.

mon, 19-nov-2012, 19:55

Footprints frozen in the Creek

Reading today’s weather discussion from the Weather Service, they state:

LITTLE OR NO SNOWFALL IS EXPECTED IN THE FORECAST AREA FOR THE NEXT WEEK.

The last time it snowed was November 11th, so if that does happen, it will be at least 15 days without snow. That seems unusual for Fairbanks, so I checked it out.

Finding the lengths of consecutive events is something I’ve wondered how to do in SQL for some time, but it’s not all that difficult if you can get a listing of just the dates where the event (snowfall) happens. Once you’ve got that listing, you can use window functions to calculate the intervals between dates (rows), eliminate those that don’t matter, and rank them.

For this exercise, I’m looking for days with more than 0.1 inches of snow where the maximum temperature was below 10°C. And I exclude any interval where the end date is after March. Without this exclusion I’d get a bunch of really long intervals between the last snowfall of the year, and the first from the next year.

Here’s the SQL (somewhat simplified), using the GHCN-Daily database for the Fairbanks airport station:

```SELECT * FROM (
SELECT dte AS start,
LEAD(dte) OVER (ORDER BY dte) AS end,
LEAD(dte) OVER (ORDER BY dte) - dte AS interv
FROM (
SELECT dte
FROM ghcnd_obs
WHERE station_id = 'USW00026411'
AND tmax < 10.0
AND snow > 0
) AS foo
) AS bar
WHERE extract(month from foo.end) < 4
AND interv > 6
ORDER BY interv DESC;
```

Here’s the top-8 longest periods:

Start End Days
1952‑12‑01 1953‑01‑19 49
1978‑02‑08 1978‑03‑16 36
1968‑02‑23 1968‑03‑28 34
1969‑11‑30 1970‑01‑02 33
1959‑01‑02 1959‑02‑02 31
1979‑02‑01 1979‑03‑03 30
2011‑02‑26 2011‑03‑27 29
1950‑02‑02 1950‑03‑03 29

Kinda scary that there have been periods where no snow fell for more than a month!

Here’s how many times various snow-free periods longer than a week have come since 1948:

Days Count
7 39
10 32
9 30
8 23
12 17
11 17
13 12
18 10
15 8
14 8

We can add one more to the 8-day category as of midnight tonight.

Meta Photolog Archives