Parente's Mindtrove

Kicking the Tires: Bluemix Insights for Weather

April 14, 2016

This post comes from a Jupyter notebook I wrote to help a colleague learn how to access a Bluemix service from Python. Along the way, I learned about pandas.io.json.json_normalize and how great it is at turning nested JSON structures into flatter DataFrames. (It deserves a short post of its own.)


In this notebook, we're going to poke at the Bluemix Insights for Weather service from Python. We'll look at what kinds of queries we can make and do a few basic things with the data. We'll keep a running commentary that can serve as an introductory tutorial for developers who want to go off and build more sophisticated apps and analyses using the service.

Get some handy libs

Let's start by getting some handy Python libraries for making HTTP requests and looking at the data. You can install these with typical Python package management tools like pip install <name> or conda install <name>.

In [1]:
# comes with Python
import json

# third-party
import requests
from requests.auth import HTTPBasicAuth
import geocoder
import pandas as pd

Get credentials

Next, we need to provision an Insights for Weather service instance on Bluemix and get the access credentials. We can follow the instructions about Adding Insights for Weather to your application to do so.

To keep our credentials out of this notebook, we can copy them from the Bluemix UI put them in a weather_creds.json file alongside this notebook. The credentials JSON should look something like the following.

{
  "credentials": {
    "username": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "password": "yyyyyyyyyy",
    "host": "twcservice.mybluemix.net",
    "port": 443,
    "url": "https://xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:[email protected]"
  }
}

Now we can load that file into memory without showing its contents here.

In [2]:
with open('weather_creds.json') as f:
    creds = json.load(f)

Try a request

Now we're ready to try a request against the REST API. We'll do this using the requests Python package.

Note: At the time of this writing, the paths documented in the REST API section of the docs are incorrect. Remove the `/geocode` section of the path and it should work. The other samples and API references in the docs are correct.

First we need to build a basic auth object with our service username and password.

In [3]:
auth = HTTPBasicAuth(creds['credentials']['username'], 
                     creds['credentials']['password'])

Then we can build the base URL of the service.

In [4]:
url = 'https://{}/api/weather/v2'.format(creds['credentials']['host'])
url
Out[4]:
'https://twcservice.mybluemix.net/api/weather/v2'

The API documentation says we need to pass latitude and longitude coordinates with our queries. Instead of hardcoding one here, we'll use the Google geocoder to get the lat/lng for my hometown.

In [5]:
g = geocoder.google('Durham, NC')
g
Out[5]:
<[OK] Google - Geocode [Durham, NC, USA]>

With these in hand, we'll build up a dictionary which requests will convert into URL query args for us. Besides the geocode, we'll include other parameters like the desired unit of measurement and language of the response. (These are all noted in the API docs.)

In [6]:
params = {
    'geocode' : '{:.2f},{:.2f}'.format(g.lat, g.lng),
    'units': 'e',
    'language': 'en-US'
}

Let's make the request to one of the documented resources to get the 10 day forecast for Durham, NC. We pass the query parameters and our basic auth information.

In [7]:
resp = requests.get(url+'/forecast/daily/10day', params=params, auth=auth)

We can sanity check that the response status is OK easily. If anything went wrong with our request, the next line of code will raise an exception

In [8]:
resp.raise_for_status()

Look at the response

If we made it this far, our call was successful. Let's look at the results. We parse the JSON body of the response into a Python dictionary.

In [9]:
body = resp.json()

Let's see what keys are in the body without printing the whole thing.

In [10]:
body.keys()
Out[10]:
dict_keys(['forecasts', 'metadata'])

And let's take a closer look at the forecasts. Instead of printing the raw, potentially nested Python dictionary, we'll ask pandas to take a crack at it and give us a nicer table view of the data in our notebook.

In [11]:
df = pd.io.json.json_normalize(body['forecasts'])

How many columns do we have?

In [12]:
len(df.columns)
Out[12]:
123

Quite a few. Let's make sure pandas shows them all.

In [13]:
pd.options.display.max_columns = 125

And now we can emit the entire thing as a nice HTML table for inspection.

In [14]:
df
Out[14]:
blurb blurb_author class day.accumulation_phrase day.alt_daypart_name day.clds day.day_ind day.daypart_name day.fcst_valid day.fcst_valid_local day.golf_category day.golf_index day.hi day.icon_code day.icon_extd day.long_daypart_name day.narrative day.num day.phrase_12char day.phrase_22char day.phrase_32char day.pop day.pop_phrase day.precip_type day.qpf day.qualifier day.qualifier_code day.rh day.shortcast day.snow_code day.snow_phrase day.snow_qpf day.snow_range day.subphrase_pt1 day.subphrase_pt2 day.subphrase_pt3 day.temp day.temp_phrase day.thunder_enum day.thunder_enum_phrase day.uv_desc day.uv_index day.uv_index_raw day.uv_warning day.vocal_key day.wc day.wdir day.wdir_cardinal day.wind_phrase day.wspd day.wxman dow expire_time_gmt fcst_valid fcst_valid_local lunar_phase lunar_phase_code lunar_phase_day max_temp min_temp moonrise moonset narrative night.accumulation_phrase night.alt_daypart_name night.clds night.day_ind night.daypart_name night.fcst_valid night.fcst_valid_local night.golf_category night.golf_index night.hi night.icon_code night.icon_extd night.long_daypart_name night.narrative night.num night.phrase_12char night.phrase_22char night.phrase_32char night.pop night.pop_phrase night.precip_type night.qpf night.qualifier night.qualifier_code night.rh night.shortcast night.snow_code night.snow_phrase night.snow_qpf night.snow_range night.subphrase_pt1 night.subphrase_pt2 night.subphrase_pt3 night.temp night.temp_phrase night.thunder_enum night.thunder_enum_phrase night.uv_desc night.uv_index night.uv_index_raw night.uv_warning night.vocal_key night.wc night.wdir night.wdir_cardinal night.wind_phrase night.wspd night.wxman num qpf qualifier qualifier_code snow_code snow_phrase snow_qpf snow_range stormcon sunrise sunset torcon
0 None None fod_long_range_daily NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN Thursday 1460664024 1460631600 2016-04-14T07:00:00-0400 Waxing Gibbous WXG 8 NaN 40 2016-04-14T13:07:47-0400 2016-04-14T02:22:10-0400 Partly cloudy. Lows overnight in the low 40s. Tonight 32 N Tonight 1460674800 2016-04-14T19:00:00-0400 None 65 29 2900 Thursday night A few clouds. Low around 40F. Winds ENE at 5 t... 1 P Cloudy Partly Cloudy Partly Cloudy 0 rain 0 None None 45 Partly cloudy 0 Partly Cloudy 40 Low around 40F. 0 No thunder Low 0 0 0 D2:DA02:X3000300023:S300021:TL40:W03R02 40 61 ENE Winds ENE at 5 to 10 mph. 7 wx1650 1 0 None None 0 None 2016-04-14T06:43:06-0400 2016-04-14T19:49:09-0400 None
1 None None fod_long_range_daily Friday 56 D Tomorrow 1460718000 2016-04-15T07:00:00-0400 Very Good 9 65 30 3000 Friday Intervals of clouds and sunshine. High 67F. Wi... 2 P Cloudy Partly Cloudy Partly Cloudy 0 rain 0 NaN NaN 40 Times of sun and clouds 0 Partly Cloudy 67 High 67F. 0 No thunder Very High 8 7.95 0 D3:DA14:X3000300031:S300033:TH67:W02R03 40 55 NE Winds NE at 10 to 15 mph. 11 wx1100 Friday 1460664024 1460718000 2016-04-15T07:00:00-0400 Waxing Gibbous WXG 9 67 39 2016-04-15T14:05:00-0400 2016-04-15T03:05:43-0400 Mix of sun and clouds. Highs in the upper 60s ... Friday night 56 N Tomorrow night 1460761200 2016-04-15T19:00:00-0400 None 61 29 2900 Friday night A few clouds from time to time. Low 39F. Winds... 3 P Cloudy Partly Cloudy Partly Cloudy 0 precip 0 None None 48 Partly cloudy 0 Partly Cloudy 39 Low 39F. 0 No thunder Low 0 0 0 D4:DA15:X3000300042:S300042:TL39:W02R02 41 53 NE Winds NE at 5 to 10 mph. 7 wx1650 2 0 None None 0 None 2016-04-15T06:41:46-0400 2016-04-15T19:49:59-0400 None
2 None None fod_long_range_daily Saturday 52 D Saturday 1460804400 2016-04-16T07:00:00-0400 Excellent 10 69 30 3000 Saturday Intervals of clouds and sunshine. High 71F. Wi... 4 P Cloudy Partly Cloudy Partly Cloudy 0 rain 0 NaN NaN 37 Mix of sun and clouds 0 Partly Cloudy 71 High 71F. 0 No thunder High 7 7.37 0 D5:DA16:X3000300031:S300032:TH71:W02R03 41 37 NE Winds NE at 10 to 15 mph. 10 wx1100 Saturday 1460664024 1460804400 2016-04-16T07:00:00-0400 Waxing Gibbous WXG 10 71 42 2016-04-16T15:01:38-0400 2016-04-16T03:44:32-0400 Times of sun and clouds. Highs in the low 70s ... Saturday night 9 N Saturday night 1460847600 2016-04-16T19:00:00-0400 None 64 29 2900 Saturday night Partly cloudy skies. Low 42F. Winds light and ... 5 P Cloudy Partly Cloudy Partly Cloudy 0 rain 0 None None 57 Partly cloudy 0 Partly Cloudy 42 Low 42F. 0 No thunder Low 0 0 0 D6:DA17:X3000320041:S300043:TL42:W9902 44 46 NE Winds light and variable. 4 wx1650 3 0 None None 0 None 2016-04-16T06:40:26-0400 2016-04-16T19:50:49-0400 None
3 None None fod_long_range_daily Sunday 0 D Sunday 1460890800 2016-04-17T07:00:00-0400 Excellent 10 71 32 3200 Sunday Mainly sunny. High 73F. Winds NNE at 5 to 10 mph. 6 Sunny Sunny Sunny 0 rain 0 NaN NaN 37 Sunny 0 Sunny 73 High 73F. 0 No thunder Very High 8 8.07 0 D7:DA04:X3200320034:S320032:TH73:W01R02 44 31 NNE Winds NNE at 5 to 10 mph. 9 wx1000 Sunday 1460664024 1460890800 2016-04-17T07:00:00-0400 Waxing Gibbous WXG 10 73 44 2016-04-17T15:57:12-0400 2016-04-17T04:19:17-0400 Abundant sunshine. Highs in the low 70s and lo... Sunday night 0 N Sunday night 1460934000 2016-04-17T19:00:00-0400 None 66 31 3100 Sunday night Clear skies. Low 44F. Winds light and variable. 7 Clear Clear Clear 0 rain 0 None None 54 Clear 0 Clear 44 Low 44F. 0 No thunder Low 0 0 0 D8:DA05:X3200320041:S320044:TL44:W9902 46 36 NE Winds light and variable. 5 wx1550 4 0 None None 0 None 2016-04-17T06:39:07-0400 2016-04-17T19:51:39-0400 None
4 None None fod_long_range_daily Monday 0 D Monday 1460977200 2016-04-18T07:00:00-0400 Very Good 9 80 32 3200 Monday A mainly sunny sky. High 82F. Winds light and ... 8 Sunny Sunny Sunny 0 rain 0 NaN NaN 39 Sunshine 0 Sunny 82 High 82F. 0 No thunder Very High 8 8.13 0 D9:DA06:X3200320033:S320034:TH82:W9902 46 331 NNW Winds light and variable. 4 wx1000 Monday 1460664024 1460977200 2016-04-18T07:00:00-0400 Waxing Gibbous WXG 11 82 54 2016-04-18T16:51:43-0400 2016-04-18T04:52:14-0400 Sunny. Highs in the low 80s and lows in the mi... Monday night 9 N Monday night 1461020400 2016-04-18T19:00:00-0400 None 73 31 3100 Monday night Clear skies. Low 54F. Winds light and variable. 9 Clear Clear Clear 0 rain 0 None None 58 Mainly clear 0 Clear 54 Low 54F. 0 No thunder Low 0 0 0 D10:DA07:X3200320044:S320041:TL54:W9902 55 292 WNW Winds light and variable. 4 wx1500 5 0 None None 0 None 2016-04-18T06:37:49-0400 2016-04-18T19:52:29-0400 None
5 None None fod_long_range_daily Tuesday 44 D Tuesday 1461063600 2016-04-19T07:00:00-0400 Very Good 9 78 30 3000 Tuesday Sunshine and clouds mixed. High near 80F. Wind... 10 P Cloudy Partly Cloudy Partly Cloudy 0 rain 0 NaN NaN 43 Partly cloudy 0 Partly Cloudy 80 High near 80F. 0 No thunder Very High 8 8.18 0 D11:DA08:X3000300034:S300031:TH80:W15R02 55 347 NNW Winds NNW at 5 to 10 mph. 7 wx1100 Tuesday 1460664024 1461063600 2016-04-19T07:00:00-0400 Waxing Gibbous WXG 12 80 52 2016-04-19T17:46:07-0400 2016-04-19T05:23:58-0400 Partly cloudy. Highs in the low 80s and lows i... Tuesday night 51 N Tuesday night 1461106800 2016-04-19T19:00:00-0400 None 72 29 2900 Tuesday night Partly cloudy. Low 52F. Winds light and variable. 11 P Cloudy Partly Cloudy Partly Cloudy 0 rain 0 None None 59 Partly cloudy 0 Partly Cloudy 52 Low 52F. 0 No thunder Low 0 0 0 D12:DA09:X3000300043:S300042:TL52:W9902 53 37 NE Winds light and variable. 3 wx1600 6 0 None None 0 None 2016-04-19T06:36:32-0400 2016-04-19T19:53:19-0400 None
6 None None fod_long_range_daily Wednesday 23 D Wednesday 1461150000 2016-04-20T07:00:00-0400 Excellent 10 75 30 3000 Wednesday Partly cloudy skies. High 77F. Winds N at 5 to... 12 P Cloudy Partly Cloudy Partly Cloudy 0 rain 0 NaN NaN 38 Partly cloudy 0 Partly Cloudy 77 High 77F. 0 No thunder Very High 8 8.24 0 D13:DA10:X3000320031:S300031:TH77:W16R02 53 0 N Winds N at 5 to 10 mph. 6 wx1100 Wednesday 1460664024 1461150000 2016-04-20T07:00:00-0400 Waxing Gibbous WXG 13 77 50 2016-04-20T18:39:35-0400 2016-04-20T05:54:50-0400 Times of sun and clouds. Highs in the upper 70... Wednesday night 5 N Wednesday night 1461193200 2016-04-20T19:00:00-0400 None 69 31 3100 Wednesday night Clear skies. Low near 50F. Winds light and var... 13 Clear Clear Clear 0 rain 0 None None 54 Clear 0 Clear 50 Low near 50F. 0 No thunder Low 0 0 0 D14:DA11:X3200320041:S320044:TL50:W9902 51 159 SSE Winds light and variable. 3 wx1500 7 0 None None 0 None 2016-04-20T06:35:16-0400 2016-04-20T19:54:10-0400 None
7 None None fod_long_range_daily Thursday 8 D Thursday 1461236400 2016-04-21T07:00:00-0400 Very Good 9 78 32 3200 Thursday Sunny. High near 80F. Winds WSW at 5 to 10 mph. 14 Sunny Sunny Sunny 10 rain 0 NaN NaN 44 Abundant sunshine 0 Sunny 80 High near 80F. 0 No thunder Very High 8 8.30 0 D15:DA12:X3200320032:S320033:TH80:W11R02 52 258 WSW Winds WSW at 5 to 10 mph. 8 wx1000 Thursday 1460664024 1461236400 2016-04-21T07:00:00-0400 Full Moon F 14 80 56 2016-04-21T19:33:54-0400 2016-04-21T06:25:55-0400 Sunshine. Highs in the low 80s and lows in the... Thursday night 32 N Thursday night 1461279600 2016-04-21T19:00:00-0400 None 72 33 3300 Thursday night Clear skies with a few passing clouds. Low 56F... 15 M Clear Mostly Clear Mostly Clear 20 rain 0 None None 62 A few clouds 0 Mostly Clear 56 Low 56F. 0 No thunder Low 0 0 0 D16:DA13:X3400300043:S340041:TL56:W10R02 57 217 SW Winds SW at 5 to 10 mph. 6 wx1500 8 0 None None 0 None 2016-04-21T06:34:00-0400 2016-04-21T19:55:00-0400 None
8 None None fod_long_range_daily Friday 46 D Friday 1461322800 2016-04-22T07:00:00-0400 Very Good 9 79 30 3000 Friday Sunshine and clouds mixed. High 81F. Winds SW ... 16 P Cloudy Partly Cloudy Partly Cloudy 20 rain 0 NaN NaN 53 Mix of sun and clouds 0 Partly Cloudy 81 High 81F. 0 No thunder Very High 8 8.35 0 D17:DA14:X3000300034:S300032:TH81:W10R03 58 234 SW Winds SW at 10 to 15 mph. 10 wx1100 Friday 1460664024 1461322800 2016-04-22T07:00:00-0400 Full Moon F 15 81 59 2016-04-22T20:27:24-0400 2016-04-22T06:58:49-0400 Times of sun and clouds. Highs in the low 80s ... Friday night 38 N Friday night 1461366000 2016-04-22T19:00:00-0400 None 74 29 2900 Friday night Partly cloudy. Low 59F. Winds SW at 5 to 10 mph. 17 P Cloudy Partly Cloudy Partly Cloudy 20 rain 0 None None 66 Partly cloudy 0 Partly Cloudy 59 Low 59F. 0 No thunder Low 0 0 0 D18:DA15:X3000300043:S300042:TL59:W10R02 60 219 SW Winds SW at 5 to 10 mph. 7 wx1600 9 0 None None 0 None 2016-04-22T06:32:46-0400 2016-04-22T19:55:50-0400 None
9 None None fod_long_range_daily Saturday 27 D Saturday 1461409200 2016-04-23T07:00:00-0400 Very Good 9 80 34 3400 Saturday Mostly sunny skies. High 82F. Winds WSW at 5 t... 18 M Sunny Mostly Sunny Mostly Sunny 10 rain 0 NaN NaN 54 Plenty of sun 0 Mostly Sunny 82 High 82F. 0 No thunder Very High 8 8.40 0 D19:DA16:X3400300032:S340033:TH82:W11R02 60 245 WSW Winds WSW at 5 to 10 mph. 7 wx1000 Saturday 1460664024 1461409200 2016-04-23T07:00:00-0400 Waning Gibbous WNG 16 82 59 2016-04-23T21:21:50-0400 2016-04-23T07:33:50-0400 Plenty of sun. Highs in the low 80s and lows i... Saturday night 17 N Saturday night 1461452400 2016-04-23T19:00:00-0400 None 74 33 3300 Saturday night Mainly clear. Low 59F. Winds light and variable. 19 M Clear Mostly Clear Mostly Clear 20 rain 0 None None 68 A few clouds 0 Mostly Clear 59 Low 59F. 0 No thunder Low 0 0 0 D20:DA17:X3400320042:S340041:TL59:W9902 60 226 SW Winds light and variable. 5 wx1500 10 0 None None 0 None 2016-04-23T06:31:32-0400 2016-04-23T19:56:40-0400 None
10 None None fod_long_range_daily Sunday 13 D Sunday 1461495600 2016-04-24T07:00:00-0400 Very Good 9 79 32 3200 Sunday Sunny. High 81F. Winds WSW at 5 to 10 mph. 20 Sunny Sunny Sunny 10 rain 0 NaN NaN 51 Sunshine 0 Sunny 81 High 81F. 0 No thunder Very High 8 8.45 0 D21:DA04:X3200320032:S320034:TH81:W11R02 60 250 WSW Winds WSW at 5 to 10 mph. 7 wx1000 Sunday 1460664024 1461495600 2016-04-24T07:00:00-0400 Waning Gibbous WNG 17 81 57 2016-04-24T22:15:01-0400 2016-04-24T08:11:01-0400 Sunshine. Highs in the low 80s and lows in the... Sunday night 24 N Sunday night 1461538800 2016-04-24T19:00:00-0400 None 73 33 3300 Sunday night A few clouds from time to time. Low 57F. Winds... 21 M Clear Mostly Clear Mostly Clear 20 rain 0 None None 67 Mostly clear 0 Mostly Clear 57 Low 57F. 0 No thunder Low 0 0 0 D22:DA05:X3400340042:S340042:TL57:W08R02 59 190 S Winds S at 5 to 10 mph. 6 wx1500 11 0 None None 0 None 2016-04-24T06:30:19-0400 2016-04-24T19:57:30-0400 None

Of course, if we weren't in a notebook, we probably wouldn't want to do this. Rather, we'd want to filter the DataFrame or the original JSON down to whatever values we needed in our application. Let's do a bit of that now.

Look at some specific columns

Now that we have an idea of all the available columns, let's dive into a few.

One of the columns appears to be a human readable forecast. Before we show it, let's make sure pandas doesn't ellipsize the text.

In [15]:
pd.options.display.max_colwidth = 125

Now we can look at the narrative along side the day it describes.

In [16]:
df[['day.alt_daypart_name', 'narrative']]
Out[16]:
day.alt_daypart_name narrative
0 NaN Partly cloudy. Lows overnight in the low 40s.
1 Friday Mix of sun and clouds. Highs in the upper 60s and lows in the upper 30s.
2 Saturday Times of sun and clouds. Highs in the low 70s and lows in the low 40s.
3 Sunday Abundant sunshine. Highs in the low 70s and lows in the mid 40s.
4 Monday Sunny. Highs in the low 80s and lows in the mid 50s.
5 Tuesday Partly cloudy. Highs in the low 80s and lows in the low 50s.
6 Wednesday Times of sun and clouds. Highs in the upper 70s and lows in the low 50s.
7 Thursday Sunshine. Highs in the low 80s and lows in the mid 50s.
8 Friday Times of sun and clouds. Highs in the low 80s and lows in the upper 50s.
9 Saturday Plenty of sun. Highs in the low 80s and lows in the upper 50s.
10 Sunday Sunshine. Highs in the low 80s and lows in the upper 50s.

There's a few mentions of the word golf in the big table columns. Let's find those columns in particular.

In [17]:
df.columns[df.columns.str.contains('golf')]
Out[17]:
Index(['day.golf_category', 'day.golf_index', 'night.golf_category',
       'night.golf_index'],
      dtype='object')

Let's look at those alongside the day names.

In [18]:
df[['day.alt_daypart_name'] + df.columns[df.columns.str.contains('golf')].tolist()]
Out[18]:
day.alt_daypart_name day.golf_category day.golf_index night.golf_category night.golf_index
0 NaN NaN NaN None
1 Friday Very Good 9 None
2 Saturday Excellent 10 None
3 Sunday Excellent 10 None
4 Monday Very Good 9 None
5 Tuesday Very Good 9 None
6 Wednesday Excellent 10 None
7 Thursday Very Good 9 None
8 Friday Very Good 9 None
9 Saturday Very Good 9 None
10 Sunday Very Good 9 None

The day time golf category and index are interesting. Night golf is ... well ... unexplained. 🌛

How about temperatures? Let's get a summary of the values for the next ten days.

In [19]:
df[['max_temp', 'min_temp']].describe()
Out[19]:
max_temp min_temp
count 10.000000 11.000000
mean 77.400000 50.181818
std 5.274677 7.665744
min 67.000000 39.000000
25% 74.000000 43.000000
50% 80.000000 52.000000
75% 81.000000 56.500000
max 82.000000 59.000000

Try another endpoint

So far we've poked at the 10-day forecast resource. Let's try another just to see how similar / different it is. Here we'll fetch historical observations for the same location.

In [20]:
resp = requests.get(url+'/observations/timeseries/24hour', auth=auth, params=params)
resp.raise_for_status()
body = resp.json()
body.keys()
Out[20]:
dict_keys(['observations', 'metadata'])

This time, the key of interest is observations.

In [21]:
obs = pd.io.json.json_normalize(body['observations'])

Fewer columns this time. Let's poke at the blunt_phrase and `valid_time_gmt.

In [22]:
obs.columns
Out[22]:
Index(['blunt_phrase', 'class', 'clds', 'day_ind', 'dewPt', 'expire_time_gmt',
       'feels_like', 'gust', 'heat_index', 'icon_extd', 'key', 'max_temp',
       'min_temp', 'obs_id', 'obs_name', 'precip_hrly', 'precip_total',
       'pressure', 'pressure_desc', 'pressure_tend', 'qualifier',
       'qualifier_svrty', 'rh', 'snow_hrly', 'temp', 'terse_phrase', 'uv_desc',
       'uv_index', 'valid_time_gmt', 'vis', 'wc', 'wdir', 'wdir_cardinal',
       'wspd', 'wx_icon', 'wx_phrase'],
      dtype='object')
In [23]:
obs.valid_time_gmt.head()
Out[23]:
0    1460619900
1    1460621100
2    1460622300
3    1460623500
4    1460624700
Name: valid_time_gmt, dtype: int64

We can make the times more human readable. They're seconds since the epoch expressed in UTC.

In [24]:
obs['time_utc'] = pd.to_datetime(obs.valid_time_gmt, unit='s', utc=True)

Now we can check the summary of the available observations within the last 24 hours. We'll reverse them so that they're sorted newest to oldest.

In [25]:
obs[['blunt_phrase', 'time_utc']].iloc[::-1]
Out[25]:
blunt_phrase time_utc
35 Temps slightly below average. 2016-04-14 19:25:00
34 Temps slightly below average. 2016-04-14 19:05:00
33 Temps slightly below average. 2016-04-14 18:45:00
32 Seasonal temperatures. 2016-04-14 18:25:00
31 Temps slightly below average. 2016-04-14 18:05:00
30 Seasonal temperatures. 2016-04-14 17:45:00
29 Seasonal temperatures. 2016-04-14 17:25:00
28 Seasonal temperatures. 2016-04-14 17:05:00
27 Seasonal temperatures. 2016-04-14 16:45:00
26 Seasonal temperatures. 2016-04-14 16:25:00
25 Seasonal temperatures. 2016-04-14 16:05:00
24 Seasonal temperatures. 2016-04-14 15:45:00
23 Seasonal temperatures. 2016-04-14 15:25:00
22 Seasonal temperatures. 2016-04-14 15:05:00
21 Seasonal temperatures. 2016-04-14 14:45:00
20 Seasonal temperatures. 2016-04-14 14:25:00
19 Seasonal temperatures. 2016-04-14 14:05:00
18 Seasonal temperatures. 2016-04-14 13:45:00
17 Seasonal temperatures. 2016-04-14 13:25:00
16 Seasonal temperatures. 2016-04-14 13:05:00
15 Seasonal temperatures. 2016-04-14 12:45:00
14 Seasonal temperatures. 2016-04-14 12:25:00
13 Temps slightly below average. 2016-04-14 12:05:00
12 Temps slightly below average. 2016-04-14 11:45:00
11 Seasonal temperatures. 2016-04-14 11:25:00
10 Patchy fog reported nearby. 2016-04-14 11:05:00
9 Seasonal temperatures. 2016-04-14 10:45:00
8 Seasonal temperatures. 2016-04-14 10:25:00
7 Seasonal temperatures. 2016-04-14 10:05:00
6 Cooler than yesterday. 2016-04-14 09:45:00
5 Cooler than yesterday. 2016-04-14 09:25:00
4 Cooler than yesterday. 2016-04-14 09:05:00
3 Cooler than yesterday. 2016-04-14 08:45:00
2 Cooler than yesterday. 2016-04-14 08:25:00
1 Cooler than yesterday. 2016-04-14 08:05:00
0 Cooler than yesterday. 2016-04-14 07:45:00

🌼 I guess it's seasonal right now. 🌼

Go further

We'll stop here. What we did in this notebook is a prelude to what's possible. Here's some ideas for further experimentation:

  • Write a simple Python function (or functions) that wrap the few lines of request logic needed to query any of the API endpoints.
  • Funnel observations into a persistent store to collect them over time. Use that historical data to try to build a predictive model (e.g., using scikit-learn).
  • Combine weather data with data from other sources in domain specific notebooks or applications.

Another Read: Code Hiding on nbviewer »

When this notebook first loads on nbviewer, all of the input cells are hidden. To reveal them, click the element tag icon in the top right of nbviewer. Click it again to hide them once more.