Software apps and online services
It is much easier to build your own connected weather station now than ever before. With the Particle Electron and SparkFun Weather Shield hardware, you can assemble your weather station in seconds, and place it anywhere covered by a 2G/3G network. Data storage and programming are conveniently available in the cloud (ThingSpeak and Particle Web IDE). This allows you to monitor your home temperature while being at work, measure the humidity in your car when driving for hours in the highway, or analyze the weather data (MATLAB) as long as you have access to the internet.
This project demonstrates how to build the weather station, configure the hardware, and interact with the weather data. Examples of code and data analysis are provided as well.Build the Weather Station
The required parts are listed in the Components Section. The assembly is as follows:
- Insert the SIM card to Electron
- Attach Electron to Photon weather shield on the footprint. Make sure to line up each pin with the same name.
- Attach the antenna to Electron
- Power Electron with the battery or the adapter
- Mount the weather station outside (You may want to validate first that the weather station works well)
Note: when connecting Electron to your computer through the USB cable, make sure that the battery is attached. The USB port may not provide enough power.Configure and Retrieve Data
This section shows you how to configure the weather station (Electron) to forward data to an internet aggregator (ThingSpeak). Electron is a cellular-only device. We are using Webhooks to conserve data. Refer to this guide on how to reduce data use.
Here are the key steps:
2. Create an account on ThingSpeak, and create a channel with seven fields.
3. Install Particle CLI by following these instructions.
4. Add a Webhook in your Particle account:
- Create a JSON file with Webhook options. You can use the TSwriteall.json provided in the Code Section. Make sure to define all the fields as shown in TSwriteall.json.
- Open a command prompt and login to the CLI with your particle account
- CD to the directory where you put the JSON file
- Run the command:
particle webhook create FileName.json
- If successful, you should see output like the following:
5. Use Particle Build (Web IDE) to program Electron:
- Create a new app using the code template (see the Code section)
- Include the libaray Sparkfun_Photon_Weather_Shield_Library 1.1.2
- Verify and flash your app to Electron (ensure your Electron is connected)
Note: Call the
particle.publish function to invoke your Webhook in the Particle app. You must use the same event name as previously defined in the JSON file. In addition, you need format your publish event data as JSON. Ensure the field numbers are consistently defined in the app, JSON file, publish event data, and ThingSpeak channel. See the annotations in the following pictures.
6. Validate the data received by ThingSpeak channelAnalyze and Visualize Data
This section illustrates some ideas that you may employ to interact with your weather data. The live data is available on ThingSpeak channel 87179. All the code is written in MATLAB, which can be downloaded in the Code section. The version of MATLAB used in this project is R2016a, with ThingSpeak Support Toolbox Add-On.
You can also run MATLAB analysis and visualization on ThingSpeak by adding Apps to your channel. No installation is required.
Get Data: Retrieve Data from ThingSpeak
In order to perform analysis and visualization, all the weather data need to be read from ThingSpeak to MATLAB. We use the thingSpeakRead function to read data in all fields from our channel 87179 and store them into individual variables respectively.
Note: The maximum number of points that ThingSpeak allows to be fetched per single query is 8000. For our data rate, it corresponds to about 6 days of data. To retrieve more than that, you need read data iteratively from the channel.
Basic Display: Weather Data Summary
A quick approach to inspecting and getting a better understanding of our data is to generate some simple visualizations, in both 2D and 3D.
The 2D plots below indicate the histogram of each weather data during the past 6 days, as well as the daily average of four data (Temperature, Humidity, Pressure and Rainfall). A tip for better demonstrating the Wind Direction histogram is to use the rose function, which plots out an angle histogram rather than a bar histogram.
The 3D plots below provide a more intuitive view of the distribution of hourly-data (Temperature and Humidity) in the past 6 days. One advantage of using the 3D-Bar plot is that you are able to make clear observation on data falling into the same range of values.
Data Analysis and Visualization: Temperature
We are going to focus on analyzing the temperature data to answer the following three questions:
- How does the temperature behave on March 7th, 2016?
- What is the overall trend during the last week?
- What are the daily metrics?
How does the temperature behave on March 7th, 2016?
To answer the first question, we download the free temperature data on March 7th, 2011-2015 from Weather Underground, and import them to MATLAB.
In the first subplot, we simply show the hourly historical temperature for each year. It is hard to measure the difference by adding the current-year data to the same axis. Instead, we calculated the historical max, min, and average at each hour, and visualize them together with the temperature on March 7th, 2016. See the second subplot. We see that the current-year temperature is mostly above the historical average through the day, and hits the historical max during 17pm-21pm.
What is the overall trend during the last week?
To answer the second question, we visualize the entire temperature data (see the blue curve in the plot below). Before fitting a trendy line, we need pre-process the data to remove any missing value in temperature and time. This can be done by using logical indexing approach in MATLAB. Then we add two fitting curves, a linear line and a sum of sine equations. The former tells us that the temperature is increasing globally with time. The latter further illustrates how the temperature behaves locally on each day.
As seen from the plot above, there are local variations in the raw temperature data. Local variations may lead to inaccuracies at day level metrics. To resolve this issue, we apply an hourly smoothing filter, i.e., the filter window size is 60. The green curve in the plot above represents the smoothed data.
What are the daily metrics?
Now we are ready to answer the third question by leveraging the smoothed temperature. We plot the smoothed temperature in chunks such that each curve corresponds to a single day. In addition, we calculate and visualize the max/min (horizontal dash lines), the average (the thick dash line), and the range of variations (the band) per hour over the past week.
The analysis and visualizations in this section can be easily extended to other data if you'd like.
Further Discovery: Dew Point and Correlation
So far we have performed calculations and visualizations on a single source of data. You can combine any data retrieved from the weather station to produce other estimations. One idea is to compute the Dew Point over the past week.
The dew point is the temperature at which dew forms and is a measure of atmospheric moisture. A dew point greater than 68 °F is considered uncomfortable for human. A well-known approximation to compute the dew point is as follows (refer to this Wiki for more details):
The visualization below shows that we may feel a bit dry over most of the past week, and the dew point is increasing slightly towards the comfortable area.
Next, let's try to inspect the relationship among three data sources, Temperature, Humidity and Pressure. As we are going to fit the data, the first step is to replace the missing value in all three data sources. We use linear interpolation method to produce a set of 500 point from the known raw data. An alternative is to remove the missing value as did in previous example, but make sure the dimensions of temperature, humidity and pressure array are consistent. We plot the scatter points and the surface in 3D. The 2D view shows that the pressure reaches the max when temperature is low and humidity is high. However, humidity has very little impact on the change of pressure.
More Ideas: Weather Data Visualization
We can create intuitive visualizations to mimic some weather phenomenon, like the wind direction and speed.
Quite similar to the angle histogram, we create a compass to summarize the wind direction in the past couple of minutes. The head of the arrow indicates the wind direction, and the length of the arrow simulates the wind speed.
Note: This is a mathematical compass with North (0 degrees), West (90 degrees), South (180 degrees), and East (270 degrees).
To observe the continuous change, we use the feather function to perform a sequential visualization of the wind along the x-axis.
As a fun example, let's build a simple "weather meter" which provides a quick and better view of the weather condition.
The cone, tied at the top of the post, plays the role of both a wind flag and a temperature indicator. From the figure below, the cone is lifted towards NW, indicating that the wind blows from SE with a non-zero speed. The higher the cone is lifted, the faster the wind blows. The color of the cone indicates the feeling of the current temperature. The comfort scale is given on the color bar to the right.
You can improve this weather meter by adding more features. For instance, by using a while loop to create a live animation of the weather condition.