Before I begin troubleshooting an issue, I always brainstorm a list of at least 2-3 things that I think may be causing the problem. In no particular order, some of the potential causes I came up with were:

• An electrical problem that is causing voltages to go haywire
• A malfunctioning or defective sensor
• A bug in the Python code on the Raspberry Pi that reads the raw data from the sensor and converts it into Celsius

Start with the Easiest Issue to Troubleshoot

Before diving into troubleshooting, I always see if there are any causes I can reasonably eliminate. In this case, a malfunctioning or defective sensor is highly unlikely to be causing the problem. The sensor is an MPL3115A2 I2C digital barometric, temperature, and altitude sensor. It has been in service on the Raspberry Pi since last May and has taken nearly 1.2 million measurements. Of those 1.2 million measurements, the QA/QC algorithm has flagged 590 of them. That works out to 0.04% of measurements being bad, or a 99.96% measurement accuracy rate.

Identify Possible Bugs where the Raspberry Pi Interprets the Sensor Readings

Of the remaining two possibilities, I always start with whatever is easiest to troubleshoot. In this case, it would be a possible bug in the Python code. On the Raspberry Pi, Python interprets sensor readings and logs them in the database. Intermittent electrical problems are a nightmare to diagnose in the best of situations. The first thing to do is to look at the data and try to identify any patterns. If you’ve ever watched any detective shows on TV, you know what they always say: evidence doesn’t lie.

One pattern should jump out at you immediately. As air temperatures go to, and drop below freezing (0°C), the sensor readings on the Raspberry Pi go haywire. It is also important to note here that this does not rule out an electrical problem causing the problem. To strengthen our case, let’s query the sensor database to see if the sensor recorded any temperatures below freezing. We want to see no instances of the sensor recording temperatures below freezing.

We can now say with 100% certainty that something happens when the temperature hits freezing. However, we still don’t know what. To fully rule out an electrical issue causing the problem, it’s time to get on Google and research how the sensor works.

Can We Fully Rule Out an Electrical Issue Between the Raspberry Pi and the Sensor?

According to the manufacturer, there are three pieces of information that will help us here.

1. The sensor reads and transmits raw data in hexadecimal format.
   • In traditional decimals (such as 0.625), each element of the decimal can store values from 0 to 9. Those values are represented by the corresponding numerals.
   • In hexadecimal format, each element of the decimal can store values from 0 to 15. Those values are represented by the numbers 0 to 9 and the letters A to F.
   • Python uses standard mathematics to convert between hexadecimal and traditional decimal formats and vice versa.
2. Because of the way hexadecimal format works, the sensors cannot handle signed (i.e. negative) values in their raw format.
3. The sensors come with a software module that converts the raw hexadecimal data to a “human-readable” decimal temperature, in Celsius.

The Problem Lies with How Digital Sensors Handle Negative Values

To finally be able to rule out an electrical problem, let’s focus on item #2. The sensors not being able to handle signed (negative) values in their raw format. With any kind of digital sensor, if it gets a reading that’s off the scale, it loops around to the opposite end of the scale.

For example, if a sensor had a scale of 0 to 400 and got a reading of -1, it would output a value of 399. A reading of -5 would output 395. It works on the other end of the scale, too. A reading of 410 would output a value of 10. During my meteorology studies at the University of Oklahoma, we routinely observed this phenomenon while measuring wind speeds inside of tornadoes with doppler radar.

Item #2 is such a key piece of the puzzle because the problem occurs as soon as temperatures drop below 0°C. With this information, we can firmly establish the bottom end of the temperature sensor’s scale. Now all we had to do was figure out what the top of that scale was. For that, we have to look at the Python code, in which I found the following equation:

raw_temperature = ((raw_temperature_msb * 256) + (raw_temperature_lsb & 0xF0)) / 16

In this line of code, Python converts the raw temperature reading to standard decimal. Python reads in the sensor’s raw hexadecimal readings bit-by-bit before converting them to Celsius and sending the to the Raspberry Pi. Each reading from the sensor contains a maximum of 2 bits. For example, if this formula was used on a standard Base 10 number, such as 27, it would be read as:

temperature = (10 * 2) + (1 * 7) = 20 + 7 = 27

Calculating the Sensor’s Upper Bound

While that information may not seem significant on its own, it gives us enough information to calculate the upper end of the sensor’s temperature scale. Remember in hexadecimal, each bit can contain a value up to 16, so calculating the upper bound (in degrees Celsius) is simply:

Upper Bound = 16 * 16 = 256

Now that we have established that the sensor has a raw output range of 0 to 256°C, let’s look back at the query results. Keep in mind what I mentioned above about off-the-scale readings looping around to the other end of the scale.

What is the Actual Temperature of the Elevated Sensor Readings

To calculate the actual temperature, simply subtract 256 from the sensor readings in the 250’s. For example, if you look at the last entry in the above table, the actual temperature would be:

Actual Temp = 254.625 – 256 = -1.375°C

Using the above query as an example, the correct actual temperatures using the loop-around equation would be:

Implementing the fix before the sensor readings even reaches the QA/QC algorithm on the Raspberry Pi is simple. We simply need to add an if statement to the Python code that converts the raw hexadecimal sensor readings to degrees Celsius. Here is the original code:

try:
    raw_temperature = ((raw_temperature_msb * 256) + (raw_temperature_lsb & 0xF0)) / 16
    unrounded_temperature = raw_temperature / 16
except:
    raw_temperature = None
    unrounded_temperature = None

With the if statement added to fix the problem, the above block of code simply becomes:

try:
    raw_temperature = ((raw_temperature_msb * 256) + (raw_temperature_lsb & 0xF0)) / 16
    unrounded_temperature = raw_temperature / 16
    if unrounded_temperature > 70:
        unrounded_temperature -= 256
except:
    raw_temperature = None
    unrounded_temperature = None

Maintain Data Integrity

The final piece of the puzzle is to update the bad data points in the database to ensure data integrity. Amazingly, we can do that with a simple UPDATE command so we don’t lose any data. Full disclosure, this is not the actual database structure in the weather station. It’s just worded that way to make it easier to understand.

UPDATE `sensor_measurement`
SET `measurement_value` = `measurement_value` - 256
WHERE `measurement_type` = 'temperature'
AND `measurement_value` >= 70;

Well, this has certainly been an interesting one. It’s always a relief to confirm that the sensor is functioning exactly as it should and the problem is nothing more than an easy-to-fix bug in the Python code on the Raspberry Pi. Until next time.

The post Digging Deeper: Diagnosing My Raspberry Pi Temperature Sensor’s “Hot Flashes” appeared first on Matthew Gove Blog.

I recently logged onto my weather station to check some high and low temperatures for the month of January. While I was casually scrolling through the data, I caught something out of the corner of my eye. When I looked a little closer, I had to do a double take.

What was even more impressive were the “heat indices”. They make summers on the Persian Gulf, where heat indices can reach ridiculous levels, look like absolute zero. For comparison, the temperature of the surface of the sun is 9,941°F.

Hunting Down Bad Data Points on the Raspberry Pi

Both my and several friends’ initial reaction was to cue the jokes: “Well, it does get hot in Arizona…”, but unfortunately on a logic and reasoning test, science will beat humor every single time. Time to figure out why this happened. First, we need to run a query for all raw temperature data my sensors recorded that were greater than 60°C (140°F). I chose that cutoff based on the hottest temperature ever recorded in Arizona. On June 29, 1994, Lake Havasu City topped out at a sizzling 53°C (128°F).

Amazingly, the query returned almost 300 hits. Here is a small sample of them.

Sensors getting screwy readings like this is part of the deal when you operate any kind of data logger like this. I am much more concerned that so many bad data points managed to slip through my QA/QC algorithm on the Raspberry Pi. I’ll admit the QA/QC algorithm was the very basic one I wrote to just “get things going”. It was long overdue for an upgrade, but still, it should have caught these.

Once I queried the dates where these bad data points occurred, the culprit was revealed.

Recently Replacing the Sensor Adds a New Wrinkle

You may recall that this past December, I had to replace the analog temperature and humidity sensor that broke. I formally decommissioned the broken sensor on December 14, 2019. Did you happen to notice the first date of bad data? That’s not a coincidence.

So what caused the QA/QC algorithm to nod off and miss these bad data points? The answer goes back to the broken analog sensor. The broken sensor measured both temperature and humidity. When the relative humidity reading hiccuped, often showing values greater than 3,000% when it did, the corresponding temperature reading would get thrown off by about 10-20°C.

The problem is that 99% of those bad temperature readings were between -5 and 15°C (23 to 59°F). During the winter months, we see actual temperatures inside that range every day here in the Phoenix area, so you can’t simply filter them out. I wrote the original QA/QC algorithm to flag relative humidity values that were greater than 100% or less then 0%. I would deal with the temperature parameter when I updated the algorithm.

The New Temperature Sensor

The new digital sensor I installed only measures altitude, barometric pressure, and temperature. As a result, the Raspberry Pi reverted to obtaining its humidity data from the National Weather Service. The NWS data is already QA/QC’d. Because my original QA/QC algorithm only flagged humidity and not temperature, it deemed every data point that passed through it “OK”, thus rendering the algorithm redundant.

To confirm that this is the issue, the database puts the data that the QA/QC algorithm flags into a separate table in the sensor database. I use that data for troubleshooting and making improvements to the algorithm. A simple query will reveal the dates of temperatures I have flagged. If swapping the sensors did in fact make the QA/QC algorithm redundant, the query will only return dates on or after the sensor replacement. I replaced the sensor on December 14, 2019.

Thankfully, fixing the issue requires nothing more than a few lines of code to add an if statement to the algorithm so it flags temperatures that are outside of an acceptable range of -20 to 60°C. (-4 to 140°F). I chose the upper limit based on the hottest temperature ever recorded in Arizona (53°C/128°F). At the other end of the spectrum, I based the lower bound off of the coldest temperature ever recorded in Phoenix (-9°C/16°F). I will tweak that range as needed.

Looking Ahead

My goal is to continually add small upgrades and fixes to the QA/QC algorithm over the next year. By the time I have the complete network of sensors up and running, it will be up to a level of complexity that is acceptable for a hobby weather station. At the same time, I want it to be held as close to professional standards as I can. Stay tuned for future posts where we will look closer at what happens in the data logger’s electrical system to cause such wacky temperature readings.

The post Houston, I Think There’s a Bug in My Weather Station’s QA/QC Algorithm appeared first on Matthew Gove Blog.

Over the summer, I noticed something odd after a monsoon thunderstorm had gone through. The outdoor relative humidity on my weather station read about 20% higher than the outdoor humidity on my thermostat. I wrote it off as an isolated incident related to the monsoon storm and didn’t think much of it.

As the summer progressed, I noticed the same thing would happen whenever a monsoon thunderstorm would pass over us. The weather station would report the relative humidity about 20% higher than it actually was. By the end of the summer, it was reading 20% high all the time, even in sunny weather.

A Possible Hack that Would End in Disaster

On several occasions, the idea popped into my head to just go into the weather station’s Python code and subtract 20% off of the relative humidity values the sensor was reading. Trust me, it was very tempting. I had to keep telling myself that doing so would open up a whole new set of problems.

What happens if the sensor starts working properly and gives a reading below 20%. I wrote the Python code. I know that negative relative humidity values will crash the system. When you calculate the dewpoint, you would have to take the natural log of a negative number, which is undefined.

It turned out that my instinct to not tinker with the source code was correct. Over the course of the fall, that 20% gap between the sensor’s readings and the actual humidity grew to 30%, then 40% and 50%. By early November, the weather station would be reporting 80-90% relative humidity when actual humidity values were in the 10-15% range. That would have made a big, ugly mess of the source code, and would not get me any closer to fixing the problem.

The Sensor Finally Kicks the Bucket

By the 1st of December, the sensor would only give humidity readings of 100%, regardless of what the actual humidity was. Pulling the raw sensor data from the main database on the Raspberry Pi confirmed this. While I pondered my next move, I changed the weather station’s Python code. It would now get relative humidity data from the National Weather Service instead of the sensor.

A Deeper Analysis of the Raspberry Pi Humidity Sensor Data

Looking at the above table piqued my interest enough to dump all of the relative humidity data out of the database that contains all of the sensor data and take a look at it. The sensor went into service in early May and takes readings once per minute, so the query returned about 300,000 data points. After crashing Microsoft Excel’s plotting tool several times (I have an old, cranky PC), I adjusted the query to only return the relative humidity values at the top of every hour.

Now, before I show you the marked up plot explaining everything, consider the following. Think of how it can help show when the humidity sensor went off the rails.

• The summer monsoon this year was almost non-existent, with only a small handful of storms the entire monsoon season
• Relative humidities on non-rainy summer days generally range from a minimum of 10-20% to a maximum of 35-45%.
• The fall was very dry as well, until a series of winter storms started impacting the desert southwest on Thanksgiving Day (November 28th).
• I also queried the weather station’s main database for days where measurable precipitation (1 mm or more) fell. This is the result:
  

When you put everything together, it should become clear where the sensor goes awry.

How Do We Fix the Sensor?

Unfortunately, the only remedy at this point is to replace the sensor, preferably one of much higher quality. The old sensor that broke was a cheap analog one I paid about $5 for. You get what you pay for, right? Thankfully, the pressure sensor, which is a digital I2C sensor, also measures temperature. With some minor rewiring, I can simply swap them out.

The nice thing about I2C sensors is that you can connect multiple sensors to the system using only one data cable, so adding additional sensors is easy at either end of the wire. I can add them either in the solar radiation shield or in the waterproof housing that houses the Raspberry Pi and the power supply. Additional sensors will get connected where the four gray wire nuts are in the above picture. I will definitely be adding an I2C humidity sensor to the Raspberry Pi and likely more as well. Stay tuned.

The post Troubleshooting a Raspberry Pi Sensor Gone Awry appeared first on Matthew Gove Blog.

A Word on Solar Radiation Shields

The purpose of a solar radiation shield is simple. Block solar radiation from reaching temperature sensors so you can get accurate readings even if you do not have a location in the shade to mount your sensor.

The goal of solar radiation shields is to ensure temperature readings that are within the accuracy for your sensor’s calibration. They block the solar radiation and allow for ample air flow across the sensor.

Experiment Design

To truly test which solar radiation shield worked best, I wanted to expose them to the most extreme environment possible. Thankfully summers here in Phoenix are more than happy to oblige. As an added bonus, they get exposed to direct sunlight in the early mornings and late afternoons for about 6 to 8 weeks on each side of the summer solstice.

Running the experiment in early June would satisfy both requirements: extreme heat/dry and exposure to direct sunlight. June is typically the hottest and driest month in Phoenix. Afternoon highs pretty routinely hit 110°F (about 45°C). Relative humidity typically bottoms out between 5% and 15%.

I took hourly observations from the temperature sensors in both the bowls and the plates and compared them to the observations at Phoenix Sky Harbor International Airport. Sky Harbor is the official National Weather Service observation station for the Phoenix area. I also compare them to the observations at Luke Air Force Base in Glendale, which is closer to my house. The observations were taken Monday through Thursday for three weeks.

Hypothesis

I went back and forth for a long time on which one I thought would perform better. The key to obtaining accurate temperature readings from inside a solar radiation shield is proper airflow across the sensor. Because of that, my gut was telling me that the shape of the plates would allow for slightly better airflow across the sensor. I opted to stick with my gut feeling for my official hypothesis and go with the plates.

Results

Before diving into the results, there are a few things we need to take into consideration:

• The greater Phoenix area is one of the largest urban heat islands in the world. You must account for the heat island when comparing the National Weather Service observations. The Weather Service takes observations inside the city, while my house is outside the city.
• The sensor inside the plates was not connected to a data logger. It is missing data points for where I was sleeping, cooking, not paying attention, etc.

Week 1: May 20 – 23, 2019

Despite my best efforts to run the experiment in as extreme an environment as I could get, the week of May 20th was unseasonably cool across the desert southwest. Normal highs in Phoenix this time of year are right around 40°C/100°F. As you can see from the plots, we didn’t get anywhere close to that benchmark. However, it did provide a nice contrast to the observations I took once the weather heated back up.

This dataset gave me pretty much what I was expecting:

• Both day and night observed temperatures were slightly warmer in the city than at my house.
• Instances of the homemade solar radiation shields having insufficient airflow or being in direct sunlight did not result in any major temperature anomalies.
• The plates performed slightly better during the heating of the day. They both stayed slightly below the city observations and were closer to the National Weather Service observations.
• There was no noticeable difference in the performance of the bowls and the plates at night.

Week 2: May 27 – 30, 2019

The Phoenix weather finally started warming up this week, giving some interesting results as the extreme environment returned. The beginning of the week started cool, but became hot by the time Wednesday and Thursday rolled around.

Right away, a few things really jumped out at me from this plot

• The effect of the solar radiation shields being in the sun, especially in the late afternoon (about 3 to 6 PM), is very noticeable, especially in the hotter temperatures.
  • Remember the plates and bowls (yellow and blue lines, respectively) should be just below the Sky Harbor (green) and Luke AFB (red) observations due to effects of the urban heat island.
• When the bowls/plates are in direct sunlight in the morning, they go into the shade at about 9:00 AM. Notice how the observed temperature on the blue and yellow lines temporarily drops right around 9 AM each day. Both solar radiation shields definitely feel the effect of being in direct sunlight.
• Interestingly, the difference between the bowls/plates observations and the National Weather Service observations remained pretty constant as high temperatures increased.
• Once again, the plates performed slightly better, as they were closer to the official National Weather Service observations.
  • Bowls are mounted west of the plates and are in direct sunlight for slightly longer in the afternoon. However, that difference is only just a few minutes, so its effect should be minimal.
• The sensors performed as expected at night, with temperatures at my house reading slightly below the city observations.

I expect that the final week of the experiment to be the hottest of the three. It will be interesting to see if those results corroborate what we’ve seen so far.

Week 3: June 3 – 6, 2019

The final week of the experiment was by far the hottest, with afternoon temperatures finally exceeding 40°C or about 105°F. Keep Week 2’s results in mind as we look at the results of Week 3.

The patterns from Week 2 are also evident in Week 3.

• The effects of both solar radiation shields being in direct sunlight are clearly evident on all four days.
• The observed temperatures were as expected at night, with temperatures at my house being slightly lower than the city observations.
• The temporary drop in observed temperatures in the bowls and plates occurred at about 9 AM each morning when the solar radiation shields went into the shade.
• The difference between the bowls/plates and the National Weather Service observations do vary a bit more than in Week 2. This is most likely due to higher wind velocities.
• Again, the plates seemed to perform slightly better than the bowls during the day.

The Verdict

This was a fun and interesting experiment with a few curveballs. I was able to draw the following conclusions and propose solutions to problems that I discovered during the experiment.

• The plates performed slightly better than the bowls in both cooler and hotter weather.
• Both types of dinnerware were affected by being in direct sunlight in the early morning and late afternoon.
  • Thankfully the solar radiation shields are completely out of direct sunlight for close to 10 months out of the year.
• Adding fans to the solar radiation shields to ensure an ample and consistent airflow across the sensor will noticeably reduce the effects of being in direct sunlight. It likely will not eliminate them completely.
• The sensors all performed as expected at night.
• Wind velocities should be recorded as well.
• All sensors should be connected to dataloggers so I can collect data 24 hours a day.

In the future, I would like to run the experiment again in both winter and summer settings. The results of those experiments combined with the results of this experiment would then give us a definitive answer as to whether bowls or plates make the best DIY solar radiation shields.

The post DIY Solar Radiation Shield: Results of the Bowls vs Plates Experiment appeared first on Matthew Gove Blog.

• A database to store the data
• A script that takes sensor readings, QA/QC’s those readings, and logs them in the database.

From Sensor To Database

The flow of data from the sensors to the weather station’s primary database, is as follows. The primary database contains the full set of weather data. It logs data every 5 minutes. The weather station’s web interface displays those data. Data for which I do not have sensors are obtained from the National Weather Service.

1. A Python script reads data from the sensors and logs them in the data logger every minute
2. The Python script QA/QC’s the raw data read from the sensor.
3. The script puts salid or “good” data points into the database, in default metric units. It flags invalid or “bad” data points and puts them into a separate table so the primary database cannot get any “bad” data.
4. Every 5 minutes, a Python script associated with the primary database queries the database containing the sensor data. It takes the most recent set of “good” data points that fall within that 5 minute period and inserts that data into the primary database, along with the data it obtains from the National Weather Service. If no “good” data points fall within that 5 minute period or if the sensor or the data logger are offline, it obtains the data from the National Weather Service.
5. Once per day, data measured by the sensors are synced with the primary database to ensure that the primary database has the correct sensor data in the event of a network outage.

Database Design

I could easily write an entire post about database design, so I’m not going to go into too much detail of why I made the decisions to design the database the way I did. Instead, here is my list of requirements for the database:

• Uses all the standard practices of relational databases
  • Only store raw, non-calculable data from the sensors.
    • Data that can be calculated from the sensor output do not need to be in the database.
    • This database is separate from the primary database for the DIY weather station, which I described above
  • Properly normalized to third normal form
  • Easily scalable up and down
• Store information about the sensors (manufacturer, model, serial number, etc)
• Have a mechanism to handle bad data points flagged by a QA/QC system
• Be easily queryable to pull data into the primary database for the DIY weather station (more on this later in this post)

From that list of requirements, this is the EER diagram I came up with for the sensor database.

Note here that the “good” data are put into the “measurement” table, while the “bad” data flagged by the QA/QC mechanism are put into the “measurement_to_qaqc” table.

The Python Script

The Python script is where all the magic happens. Its main purpose is to QA/QC the data it reads from the sensors, essentially acting as a guard so bad data points do not get into the weather station’s primary database.

The first iteration of the QA/QC algorithm is quite simple, and is to a degree, a bit crude. Essentially all it does is to just ensure that the data are within acceptable ranges. For example, the script flags and removes relative humidity readings of 3,285%, as well as the temperature readings of -145°C.

I haven’t begun coding a version 2 of the QA/QC algorithm, but it will look at trends in the data and eliminate unrealistic spikes. For example, if the sensors show that the temperature rose 20°C over the course of a couple minutes, that would be flagged and removed by the QA/QC algorithm. In the current first version, you could in theory have temperature readings of 10°C and 30°C within a minute of each other and the algorithm would not flag it, because both temperature values are within the acceptable or realistic temperature range.

Python Classes to Parse Sensor Readings

Within the Python code, there are classes for each type (make/model) of sensor. To instantiate an object of one of these classes, they are passed the GPIO pin in the data logger to which they are connected, as well as the Sensor ID, which is the primary key in the “sensor” table in the database described above.

The sensor classes also perform the following:

• Read the raw data from the sensors and QA/QC those readings.
• Round all data readings to a maximum of 5 decimal points, which the database requires
• Inserts the data read from the sensors into the database.
• Convert raw pressure to mean sea level pressure
• Run a test measurement. The user manually calls the test method, which takes a sensor reading and dumps the data to a Terminal window without interacting with the database.

The primary script in the data logger Python module performs the following steps to log the sensor data:

1. Instantiates an instance of the appropriate sensor class for each sensor in the weather station network
2. Loops through each sensor
3. Reads the data from the sensor
4. QA/QC’s the data from the sensor
5. Inserts the “good” data into the database

Well that just about wraps up the sensor and data logger portion of the DIY weather station project. I certainly had a blast designing and building, and I hope you enjoyed reading about it. Until next time.

The post DIY Weather Station, Part 4: Database Configuration and Programming the Sensor Readings appeared first on Matthew Gove Blog.

Supplies Needed

• Raspberry Pi, with case and power adapter
• GPIO Ribbon Cable for the Raspberry Pi, which makes it easier to connect the sensors.
• MPL3115A2 I2C Barometric Pressure, Altitude, and Temperature Sensor
• AM2302 Wired Temperature/Humidity Sensor
• 22 AWG Wire Jumpers, to connect the sensors to the Raspberry Pi
• Thermostat Wire
• Breadboard for Testing (optional)

Do a Test Run Inside First

Before installing everything at the data logger site, I always do a test run in a controlled environment indoors first. I set everything up in my home office, which allowed me to spread everything out. I could confirm that I hooked everything up correctly, and easily troubleshoot any issues. As a nice bonus, it let me work in the air conditioning instead of outside in this 100-plus degree heat.

First, get the Raspberry Pi up and running. The Pi will come with instructions for how to install the Raspbian operating system, which is a distribution of Debian Linux. After installing the operating system, simply plug in an external monitor and an ethernet cable. Then plug the Raspberry Pi in to turn it on. Once it boots up, log in and make sure you can get online and open a Terminal window.

Before connecting any sensors, shut down the Pi and unplug it. Never attempt to connect any sensors to the Pi while it is powered on! If you connect them to the wrong ports, you can damage both the sensor and the Pi by either running incorrect voltage through it or by shorting it out. After shutting down and unplugging the Raspberry Pi, connect the GPIO ribbon cable. Then get your sensors and jumper wires ready.

Each sensor will have instructions for which GPIO pin it needs to be connected to. Most sensors will connect to either 3 or 4 GPIO pins. One pin feeds the hot and neutral wires for power, and the remaining pins transfer data. For testing the sensor, connect the sensor to the GPIO pin with the jumper wires. If the sensor has wires built into it, which our temperature/humidity sensor does, you can use those.

A Few Tips for Connecting the Sensors

• Most sensors support using both 3.3 Volts and 5 Volts for power. For this test (and any other application where you are using short lengths of wire), always use 3.3 Volts. You may damage the sensor if you use 5 volts. Only use 5 volts is you have long lengths of wire. Unfortunately the definition of “long” varies depending on what type of sensor you’re using.
• Each sensor will come with instructions telling you which GPIO pin to connect it to. Please be aware that if you’re using a GPIO ribbon, those pins will be a mirror image of the GPIO pin diagram that comes with the Raspberry Pi. If you’re connecting the sensor directly to the pins on the Pi’s board (i.e. not using a GPIO ribbon), that diagram will be correct.

Once you have the sensor(s) connected, it’s time to test them, so power the Pi back on.

Test the Sensor After Connecting It

Most sensors come with scripts so you can verify that they’re working. Both sensors I’m connecting to the data logger come with Python scripts for testing, which I downloaded from GitHub. After installing any dependencies those scripts require, run the test script in a Terminal window. If the sensors are connected correctly, the sensor readings will appear in the Terminal window’s output. Make sure the sensor readings are accurate, too! If you’re in the United States, be aware that nearly all test scripts will output sensor readings in metric units by default.

Tip #1: Once you have verified the sensors are working correctly, I highly recommend taking a picture of the GPIO ribbon that shows each sensor wire connections. That way you can easily re-create the setup when you move it to the permanent data logger site.

Tip #2: Before you move the Pi to the permanent data logger site, enable SSH logins and turn off the GUI so it will be command line only. I don’t have an easy way to get a monitor to the data logger site so it is much easier to just SSH into the Pi from my laptop over my local network.

Installing the System at the Data Logger Site

Now that we’ve confirmed the sensors are working, it’s time to permanently install them at the data logger site. Make sure you shut down and power off the Pi before unplugging any sensors to move them.

A Quick Note About Thermostat Wire

I chose to use thermostat wire to connect the sensors. It is made out of a high-quality, highly conductive metal (solid copper). The conductors in thermostat wire are the perfect size for connecting to the sensors and the Raspberry Pi. It comes in many different configurations (number of conductors). It’s also cost-effective and won’t break the bank.

For the DIY weather station project, I used 18/5 (18 gauge, 5 conductors inside) wire because the pressure sensor has four connections. 18/3 wire would work fine for the temperature/humidity sensor, which only has three connections. For more complex sensor configurations, I have seen thermostat wire with up to 9 conductors available at most hardware stores.

Installing the Sensors

We are connecting two sensors to the data logger here: the temperature/humidity sensor and the barometric pressure sensor. We will mount the barometric pressure sensor in the data logger housing with the Raspberry Pi. It can be connected in the same manner it was for our inside test using a short piece of thermostat wire.

The temperature/humidity sensor is a little trickier, but if you’ve made it this far, it’s nothing you can’t handle. Before diving in, remove about 8 cm (3 in) of the jacket on the thermostat wire to reveal the conductor wires inside.

Put the Sensor in its Housing Before Connecting any Wires

The first step is to mount the sensor in the solar radiation shields we created in Part 1 of the DIY weather station project. I hung a small piece of wood in the radiation shield’s cavity and screwed the sensor to that. Make sure there is a hole in the top of the radiation shield to run the thermostat wire through. The hole should be just barely big enough to fit the wire through. After attaching the sensor to the wood, feed the end of the thermostat wire through the hole in the top of the radiation shield. Connect the sensor wires to the conductors of the thermostat wire using gray wire nuts. You can cut off any unused conductors in the thermostat wire or fold them back so they’re out of the way.

Next, securely run the thermostat wire back to where the data logger will be mounted. In my setup, I ran the wire through a conduit attached to the shepherd’s pole from which the radiation shield hung, then underground and up to the data logger through the same conduit through which the power and internet cables run.

An Important Note About Voltage and Wire Length

The DIY weather station’s analog temperature sensor can only support a maximum wire length of about 3.5 meters (11.5 ft). If you cannot get a sensor reading using 3.3 volts, use 5 volts instead. You’ll likely need to use 5 volts for wire lengths greater than about 1 meter. If you still cannot get a sensor reading using 5 volts, you’ll have to shorten the wire. By contrast, the digital pressure sensor can support a wire length up to approximately 30 meters (100 ft).

Installing the Data Logger

The data logger can be mounted in the sprinkler box housing we installed on the side of the house in Part 2 of this project. Run pieces of string through holes in the bottom of the Raspberry Pi’s case to attach it to the mount points inside the sprinkler box.

After mounting the Raspberry Pi, connect the sensor wires to their corresponding GPIO pins. If you have issues connecting the thermostat wire directly to the GPIO pins, connect one end of a jumper wire to the GPIO pin and attach the other end of the jumper wire to the sensor wire with a gray wire nut. After confirming the sensor wires are in place, connect the ethernet cable and plug the Raspberry Pi into the outlet in the sprinkler box to power it on. SSH into the Pi from your computer and run the test scripts again to confirm that the sensors are working properly.

Now that the sensors and data logger are in place, in the next phase we will set up the database, program the data logger to pull data from the sensor at specific intervals, and connect the data logger to the rest of the DIY weather station.

The post DIY Weather Station, Part 3: Installing the Data Logger and Connecting the Sensors appeared first on Matthew Gove Blog.

The sensors and database for my DIY weather station are up and running in my office. It’s finally time to install them in their permanent home. The first step is to mount the housing for the datalogger on the side of the house. I chose the spot because it’s on the north side of the house, where it’s out of direct sun (except for early evenings in the summer), and it’s right next to the junction box that brings the ethernet into the house. For the housing, I used a sprinkler timer box, which are readily available at any garden center or hardware store.

The sprinkler box comes with a GFCI outlet in it, providing a weatherproof location to plug in the Raspberry Pi that will be used as a datalogger. To connect the power, you’ll need some 14/3 wire to connect the new outlet to an existing circuit. Full disclosure: this setup likely does not comply with electrical codes. Please reference your local electrical codes if you want to bring your setup up to code.

Connect the Power Source

Important: Before doing anything with electricity, always turn the power off to the circuits with which you are working at the breaker. After turning the power off, test an outlet on the circuit to confirm that the power is actually turned off. You’d be amazed at how often you think you’ve turned the power off when you actually turned off a different circuit. Failure to shut off the power can result in serious injury or death.

First, peel back the jacket on the wire to reveal the three wires inside. Then strip the ends of the wires.

Next, insert the three wires into the corresponding terminals on the outlet. For a GFCI outlet, they must go in the end marked “Line”. Remember my “B-to-B” rule: black-to-brass. The black (hot) wire, which the electricity comes in on, goes to the brass screw. The white neutral wire, which returns the electricity to the source to complete the circuit, goes to the silver screw, and the green or bare wire goes to the ground terminal. Tighten the screws and give each wire a tug to ensure they’re secure in the terminals.

Next, run the wire through the hole in the bottom of the sprinkler box and screw the outlet into place. Connect the other end of the wire to the power source. I recommend you connect the power source in the same manner as you would add an outlet to any other circuit in the house – by connecting the wire to the “Load” terminals on any existing GFCI outlet.

Run the Wires for Electrical Power

If you’re running the wire through conduits or any other channels, run them through there now. Then connect the other end of the wires to the power source. I recommend temporarily turning the power back on and testing the outlet with a GFCI tester to make sure you wired everything correctly.

Run the Ethernet Cable

We’ll run the internet cable in a very similar manner. Run it through the other hole in the bottom of the sprinkler box, and through the same path as the electrical cables to your live network or internet connection.

Once the wires are all run, bundle any access wire, secure them with zip ties, and tidy everything else up.

Finally, fill in the trench to bury the underground wires. Turn the power back on, and you’ll be ready for the next stage: installing the datalogger, connecting the sensors, and connecting it to the network. We will cover all that, and more, in the next installment of this series.

The post DIY Weather Station, Part 2: Wiring Power and Internet to the Sensors appeared first on Matthew Gove Blog.

The biggest challenge of thermometry, especially in a climate like Arizona, is obtaining accurate temperature readings when you routinely have to deal with extreme heat and intense sun. The solution is to use a solar radiation shield, which is an enclosure in which you can mount temperature sensors, which blocks the sun, but still allows ample air flow over the sensor so you can obtain accurate temperature measurements without access to shade.

Unfortunately, quality solar radiation shields that actually work can be rather expensive. Solar radiation shields for just the cheap outdoor thermometers you buy at the hardware store generally run $50 to $100. When you have multiple sensors you need to house (I have 3), it can quickly become prohibitively expensive. For avid DIY enthusiasts like me, coupled with being a cheap skate at times, building one from scratch was a no brainer.

Materials List for a DIY Solar Radiation Shield

• 5 or 6 white or light gray plastic bowls or plates. Get the cheap ones from your local dollar store. I used four 79-cent bowls from Target for a total of $3.16.
• Three 1/4-20 x 12 in threaded rods. I picked them up at Home Depot for $0.98/each.
• One 1-in U-bolt. This is used for hanging the solar radiation shield from a pole or overhang. $0.95 at Home Depot.
• At least 1 meter (3 feet) of 1/2 in PVC pipe. The pipe will be cut up and used as spacers for the bowls/plates. You can get a 10 foot piece from Lowe’s or Home Depot for $2.20.
• Three 1/4-20 lock nuts. $0.15/each at Ace Hardware.
• Three 1/4-20 hex nuts. $0.12/each at Ace Hardware.
• Six 1/4 in washers. $0.17/each at Ace Hardware

Total Cost: $10.57

Procedure

Please Note that my original design used 5 cm spacers and 5 plastic bowls, which you see in the pictures, but the system was not getting enough air flow, resulting in abnormally high temperature readings. I modified the design to use 7 cm spacers and 4 plastic bowls.

Drill Holes in the Bottom of Each Bowl

1. Using a hole saw, cut holes in the center of all but one of the bowls. Those holes should be just big enough to hold your sensor(s). The sensor for my DIY weather station is small, so I used a 2 inch (50 mm) hole.

2. Drill three 1/4 inch holes in an equilateral triangle pattern around the edges. You will eventually put the threaded rods through these holes, which hold the shield together. The easiest way to ensure they line up is to stack a drilled bowl on top of an undrilled bowl and mark the hole locations with a pencil.

3. Drill 2 holes in the remaining bowl (the one you didn’t cut with the hole saw) for the U-Bolt. Center the U-Bolt so the solar radiation shield hangs evenly. Mount the U-Bolt in these holes.

Calculate the Spacing Between Each Bowl

4. Using a saw, cut the PVC pipe into equal length sections in the length of your desired spacer (how far you want the bowls/plates apart). I cut mine into 7 cm lengths.

5. Thread a washer and one of the lock nuts about 1 inch onto the ends of each threaded rod. Make sure you thread them all on the same distance, or the solar radiation shield will not hang level.

6. Insert the three threaded rods through the holes in one of the bowls. If you’re worried about the threaded rods sticking out the top, put spacers between the lock nut and the bottom bowl. You may need a fender washer to hold the spacers in place.

Assemble Your Solar Radiation Shield

7. Put a spacer on each threaded rod, and then add the next bowl. Repeat until you’ve used all of the bowls. Make sure the bowl at the top of the solar radiation shield is the one with the U-Bolt. I recommend that you put your sensor in the shield during this assembly, as it can be difficult to mount sensors inside the shield once it’s fully assembled.

8. Once all of the bowls are in place, put a washer and a hex nut on the top of the three threaded rods and tighten the hex nut until it’s snug. Don’t tighten it too tight, as you’ll risk cracking or breaking something.

The final product should look something like this. Hang temperature sensors at a height of 1.5 meters.

And here is the one I built out of plates if you’re curious.

Stay tuned for future posts where we will be digging into how to mount and wire the sensors and data logger, as well as the results of my experiment to determine whether the plates or the bowls makes a better solar radiation shield. If you have any questions, feel free to ask in the comments section below.

The post DIY Weather Station, Part 1: Building a Solar Radiation Shield from Scratch appeared first on Matthew Gove Blog.