thu, 16-jul-2009, 15:30

Yesterday I looked at how wind might be affecting my bicycling to and from work. Today I’ll examine the idea that Miller Hill is confounding the effect of wind on average speed by excluding this portion of the trip from the analysis. To do this, I include a bounding box comparison in the SQL statement that extracts the wind factors for track points. The additional WHERE condition looks like this:

```ST_Within(point_utm, ST_SetSRID(ST_MakeBox2D(ST_Point(454861,7193973), ST_Point(458232,7199159)), 32606))
```

The same ST_Within test is used in the calculation of average speed for each of the trips from work to home. After compiling the wind factors and average speeds, we compare the two using R. Here are the updated results:

```lm(formula = mph ~ wind, data = data)

Residuals:
Min       1Q   Median       3Q      Max
-1.87808 -0.55299  0.04038  0.62790  1.19076

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  16.8544     0.2176  77.442   <2e-16 ***
wind          0.3896     0.2002   1.946   0.0683 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.9445 on 17 degrees of freedom
Multiple R-squared: 0.1822,     Adjusted R-squared: 0.1341
F-statistic: 3.788 on 1 and 17 DF,  p-value: 0.06834
```

This time around the model and both coefficients are statistically significant (finally!), and “wind factor” is positively correlated with my average speed over the part of the route that doesn’t include Miller Hill and Railroad drive. It’s not a major contributor, but it does explain approximately 18% of the variation in average speed.

thu, 16-jul-2009, 07:08

Cycling with the wind
Photo by ~BostonBill~

I decided to look at wind a little more deeply after yesterday’s bike ride home. It seemed clear to me that the wind was strongly at my back for much of the route. It wasn’t my fastest ride home, but it was close, and it didn’t feel like I was working all that hard.

Here’s the process. First, examine all my bicycling tracks individually, using PostGIS’s ST_Azimuth function to calculate the direction I was traveling at each point. The query uses another of the new window functions (lead) in PostgreSQL 8.4.

```SELECT
point_id, dt_local,
ST_Azimuth(
point_utm,
lead(point_utm) OVER (PARTITION BY tid ORDER BY dt_local)
) / (2 * pi()) *360
FROM points
WHERE tid = TID
ORDER BY dt_local;
```

Then, for each point, find the direction the wind was blowing. This is a pretty slow query, but I haven’t found a better way to compare timestamps in the database to find the closest record. This technique, based on converting both timestamps to “epoch,” which is the number of seconds since January 1st, 1970, is faster than using an interval type of operation (like: WHERE obs_dt - POINT_DT BETWEEN interval '-3 minutes' AND interval '3 minutes').

```SELECT obs_dt, wdir, wspd
FROM observations
WHERE abs(extract(epoch from obs_dt) - extract(epoch from POINT_DT)) < 5 * 60
AND wspd IS NOT NULL
AND wdir IS NOT NULL
ORDER BY abs(extract(epoch from obs_dt) - extract(epoch from POINT_DT))
LIMIT 1;
```

Now I’ve got the direction I was traveling and the direction the wind is coming from. I wrote a Python function that returns a value from –1 (wind is in my face) to 1 (wind is at my back). The procedure is to convert the wind directions to unit u and v vectors and get the distance between the endpoints of each vector. The distances are then scaled such that wind behind the direction traveled range from 0 – 1, and from –1 – 0 for wind blowing against the direction traveled.

```def wind_effect(mydir, winddir):
""" Returns a number from 1 (wind at my back) to -1 (wind in my face)
based on the directions passed in.

Remember that wind direction is where the wind is *from*, so a
wind direction of 0 and a mydir of 0 means the wind is in my face.
"""
try:
mydir = float(mydir)
winddir = float(winddir)
except:
return(None)

my_spd = 1.0
wind_spd = 1.0
u_mydir = -1 * my_spd * math.sin(math.radians(mydir))
v_mydir = -1 * my_spd * math.cos(math.radians(mydir))
u_winddir = -1 * wind_spd * math.sin(math.radians(winddir))
v_winddir = -1 * wind_spd * math.cos(math.radians(winddir))
distance = math.sqrt((u_mydir - u_winddir)**2 + (v_mydir - v_winddir)**2)
factor = (1.41421356 - distance)
if factor < 0.0:
factor = factor / -0.58578644
else:
factor = factor / -1.41421356

return(factor)
```

Finally, multiply this value by the wind speed at that time, and sum all these values for an entire bicycling track. The result is a “wind factor.” A positive wind factor means the wind was generally at my back during the ride, negative means it was blowing in my face. Yesterday’s ride home had the highest wind factor (1.07) among trips since June. So the wind really was at my back!

Can “wind factor” help predict average speed? Here’s the R and results:

```\$ R --save < wind_from_abr.R
> model<-lm(speed ~ wind, data)
> summary(model)

Call:
lm(formula = speed ~ wind, data = data)

Residuals:
Min       1Q   Median       3Q      Max
-0.90395 -0.46782 -0.04334  0.40286  0.85918

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  14.7796     0.1471  100.48   <2e-16 ***
wind          0.4369     0.2875    1.52    0.147
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.5522 on 17 degrees of freedom
Multiple R-squared: 0.1196,     Adjusted R-squared: 0.06784
F-statistic:  2.31 on 1 and 17 DF,  p-value: 0.1469
```

Hmm. Not a whole lot of help here. The model is close to being statistically significant (although it’s not…), and it’s not very predictive (only 12% of the variation in average speed is explained by wind factor). However, the directionality of the (not quite statistically significant) wind coefficient is correct. A positive wind factor is (weakly) correlated with a higher average speed.

Thinking more about my route from work, I suspect that the route is actually two trips: the trip from ABR to the bottom of Miller Hill (4.8 miles) and the two mile trip over Miller Hill to our house. I’ll bet that wind becomes statistically significant if I only consider the first part of the trip: wind doesn’t have as much effect on a hill climb, and after making it over the top, the rest is a bumpy, gravel road where speed is determined more by safety than wind or how hard I’m pedalling. I think this might also resolve the question of why the ride home is so much easier than to work. It’s not because I’m glad to be out of work or because I’m carrying a lunchbox full of food to work, it’s because it’s downhill from ABR to the bottom of Miller Hill.

sat, 11-jul-2009, 11:04

West weather sensors

Fairbanks had some very hot weather earlier this week, including breaking a high temperature record at the airport on Wednesday afternoon. We got up to 94°F here on the Creek, but what was worse was that the low temperature on Wednesday night was 60°F, too warm to cool the house much. We were tempted to sleep out in the back cabin because the house was so warm.

It’s been cooler in the last couple days. But how much cooler?

The new version of PostgreSQL (8.4) has support for window functions, which make it easier to answer questions like this. Window functions allow you to calculate aggregates (average, sum, etc.) at one level, while display the data from another level. In this case, I want to compare the hourly average temperatures over the last 24 hours with the overall hourly average temperature over the past week. Without window functions, I’d need to combine a query that yields the hourly average temperature over the last 24 hours with a query that calculates overall seven-day hourly average temperatures. And if I want the difference between the two, the first two queries become a subquery of a third query.

Here’s the query:

```SELECT dt, t_avg,
seven_day_avg::numeric(4,1),
(t_avg - seven_day_avg)::numeric(4,1) AS anomaly
FROM (
SELECT dt, t_avg::numeric(4,1),
avg(t_avg::numeric(4,1)) OVER (PARTITION BY extract(hour from dt)) AS seven_day_avg
FROM hourly WHERE dt > current_timestamp - interval ’7 days’
) AS sevenday
WHERE dt > current_timestamp - interval ’24 hours’
ORDER BY dt;
```

And the result for the last 24 hours of temperature data:

```         dt       | t_avg | seven_day_avg | anomaly
------------------+-------+---------------+---------
2009-07-10 11:00 |  67.1 |          72.3 |    -5.2
2009-07-10 12:00 |  70.8 |          75.8 |    -5.0
2009-07-10 13:00 |  72.9 |          77.4 |    -4.5
2009-07-10 14:00 |  74.1 |          78.5 |    -4.4
2009-07-10 15:00 |  74.6 |          80.2 |    -5.6
2009-07-10 16:00 |  75.9 |          80.4 |    -4.5
2009-07-10 17:00 |  76.1 |          81.0 |    -4.9
2009-07-10 18:00 |  76.9 |          80.4 |    -3.5
2009-07-10 19:00 |  76.5 |          79.3 |    -2.8
2009-07-10 20:00 |  73.1 |          77.0 |    -3.9
2009-07-10 21:00 |  69.1 |          73.5 |    -4.4
2009-07-10 22:00 |  63.7 |          68.2 |    -4.5
2009-07-10 23:00 |  57.6 |          62.3 |    -4.7
2009-07-11 00:00 |  52.1 |          56.5 |    -4.4
2009-07-11 01:00 |  48.5 |          52.6 |    -4.1
2009-07-11 02:00 |  45.5 |          49.3 |    -3.8
2009-07-11 03:00 |  43.4 |          47.9 |    -4.5
2009-07-11 04:00 |  42.2 |          47.1 |    -4.9
2009-07-11 05:00 |  44.8 |          47.6 |    -2.8
2009-07-11 06:00 |  47.5 |          49.7 |    -2.2
2009-07-11 07:00 |  51.2 |          53.8 |    -2.6
2009-07-11 08:00 |  55.3 |          59.2 |    -3.9
2009-07-11 09:00 |  60.4 |          64.0 |    -3.6
2009-07-11 10:00 |  65.5 |          68.3 |    -2.8
```

Conclusion? It’s been several degrees cooler in the last 24 hours compared with the last seven days.

And window functions are groovy.

Meta Photolog Archives