You searched for terria - Matthew Gove Blog https://blog.matthewgove.com/ Travel the World through Maps, Data, and Photography Thu, 28 Apr 2022 14:48:12 +0000 en-US hourly 1 https://wordpress.org/?v=6.1.5 https://blog.matthewgove.com/wp-content/uploads/2021/03/cropped-android-chrome-512x512-1-32x32.png You searched for terria - Matthew Gove Blog https://blog.matthewgove.com/ 32 32 Matt Gove Photo Scenic Drives https://blog.matthewgove.com/project/matt-gove-photo-scenic-drives/ https://blog.matthewgove.com/project/matt-gove-photo-scenic-drives/#respond Fri, 22 Apr 2022 22:49:23 +0000 https://blog.matthewgove.com/?post_type=portfolio_item&p=4376 As a travel, nature, and landscape photographer, I love taking scenic drives through beautiful countrysides. One of my primary long-term goals is to be able to drive these scenic backcountry roads, not just in North America, but around the world. Additionally, I want to be able to inspire you to […]

The post Matt Gove Photo Scenic Drives appeared first on Matthew Gove Blog.

]]>
As a travel, nature, and landscape photographer, I love taking scenic drives through beautiful countrysides. One of my primary long-term goals is to be able to drive these scenic backcountry roads, not just in North America, but around the world. Additionally, I want to be able to inspire you to get out and enjoy these beautiful drives as well.

As I graduated from hobbyist to semi-pro, and now to professional photographer, the need for a single, centralized location to log scenic drives is more critical than ever. The Scenic Drives map includes scenic drives I’ve taken, am hoping to take, are on the bucket list, and more.

The Scenic Drives Application and Map

Using the TerriaJS mapping software, a database, and a simple PHP back end, we built an application that holds all of the scenic drives you see on the map. When you click on a scenic drive on the map, you’ll see all of the details for that route. You’ll get the mileage, road surface, high clearance and 4×4 requirements, sunrise, sunset, and much more. At its launch in 2016, the database contained over 1,200 drives in 110 countries. We’ve only added more as time has gone on.

The Benefits

The scenic drives map application has provided countless benefits. It has proven to be an invaluable planning tool. You have the ability to query the database on location, elevation, road surface, vehicle requirements, and much more. Additionally, it has proven invaluable to share these scenic drives with friends, family, and anyone else who is interested in them.

More Maps

If the scenic drives map has left you craving more, we have plenty of additional interactive maps to explore. From our media map highlighting our photos and videos to our bucket list to travel guides and more, we’ve got maps for everyone. We are data science and GIS professionals after all.

The post Matt Gove Photo Scenic Drives appeared first on Matthew Gove Blog.

]]>
https://blog.matthewgove.com/project/matt-gove-photo-scenic-drives/feed/ 0
Travel Bloggers: How to Stand Out with Powerful Interactive Maps https://blog.matthewgove.com/2022/02/04/travel-bloggers-how-to-stand-out-with-powerful-interactive-maps/ Fri, 04 Feb 2022 16:00:00 +0000 https://blog.matthewgove.com/?p=3818 Interactive maps are one of the most powerful tools available to travel bloggers today. Unfortunately, far too many travel bloggers today are either using interactive maps incorrectly or not using them to their full potential. As a GIS expert, data scientist, and travel blogger myself, it pains me greatly when […]

The post Travel Bloggers: How to Stand Out with Powerful Interactive Maps appeared first on Matthew Gove Blog.

]]>
Interactive maps are one of the most powerful tools available to travel bloggers today. Unfortunately, far too many travel bloggers today are either using interactive maps incorrectly or not using them to their full potential. As a GIS expert, data scientist, and travel blogger myself, it pains me greatly when I see so many crappy maps on travel blogs. There is so much potential going to waste. As a result, I want to help you, the travel blogger, realize the full potential of your interactive maps. At the very minimum, you’ll see more traffic, retain more visitors, and make your website easier to navigate.

What Makes a Good Interactive Map?

Regardless of what industry you’re in, a good interactive map is easy to understand, offers an intuitive user experience, and draws the user’s focus to the data on the map. One of the best examples of an interactive map is the RadarScope application, which we covered in detail last month in 6 Powerful Weather Applications for Stunning Landscape Photography. If you’re not familiar with it, RadarScope is an application that plots weather radar data, severe weather warnings, and much more on a map.

So how exactly does RadarScope do it so well? When I look at the maps in the screenshots above, I make a few key observations about what makes it such a powerful interactive map, even without having access to its interactivity.

  1. Your eye is immediately drawn to the radar data on the screen.
  2. The basemap is simple enough that you instantly know where the storms are, while at the same time fading into the background and not distracting you from the data on the map.
  3. RadarScope’s chosen color scheme is easy to understand because it’s the industry standard for the weather and meteorology field.
  4. The design of RadarScope’s user interface is timeless. The last two screenshots in the above gallery are from the El Reno (31 May) and Moore, Oklahoma (20 May) tornadoes in 2013. The remaining screenshots were taken between 2017 and 2021.

You Can Bring That Same User Experience to Your Travel Blog Without Breaking the Bank

Thankfully, there are plenty of tools and applications available today to bring a similar mapping experience to your travel blog. Best of all, many are free and open source, so you don’t have to spend your hard-earned cash on expensive licensing fees. However, before we look at solutions, let’s have a look at the problem.

Too Many Travel Bloggers’ Interactive Maps are Not Actually Fully Interactive

In order to fully understand the problem so many travel bloggers run into, let’s first look at the definition of interactive.

Interactive: allowing a two-way flow of information between a computer and a computer-user; responding to a user’s input.

Oxford Dictionary

Armed with that definition, have a look at what you’ll find on far too many travel blogs. From the home page, you click on a link or button to the interactive map. You see a simple map that looks something like this screenshot. Some of the countries may be shaded to indicate that the blogger has traveled there.

A Map of world with countries shaded is not an interactive map
You’ll find maps that look like this on a lot of travel blogs

If you hover over a country, it’ll often show the country’s name and maybe a count of the number of blog posts, photos, or videos the travel blogger has created. However, if you click on a country, you’ll just be brought to another page with a list of post titles. In the best case scenario, you’ll also see a featured image and the first 20-40 words of the post, much like our All Posts page.

However, this still leaves me asking one question. If you have a map whose only purpose is to redirect visitors off of said map, why even have the map at all?

How Can Travel Bloggers Make Their Maps Interactive?

First and foremost, if you’re going to add a feature to your blog or website, it should serve more of a purpose than just redirecting visitors off of it. Your visitors should be able to gather all of the information they need without leaving the map. There is one exception, however. If you’re trying to display lengthy content, such as an entire blog post, don’t try to put the entire post in a pop-up window on the map. Nobody in their right mind is going to scroll through all of that.

Instead, you want to include key details and a link to the full blog post to make a fully interactive map. For a blog post, you’ll want to include at least four things.

  • Title of the post
  • A summary of the post or the first 20-25 words of it
  • The post’s feature image
  • A link to the full post
  • Anything else that’s relevant and important, such as the date, author, or location

Example: The Matt Gove Photo Visual Media Map

To best demonstrate a fully interactive map, let’s have a look at the Matt Gove Photo Visual Media Map. Instead of blog posts, the map includes datasets to display all of our photos and videos on a map. Notice how it includes all five requirements for full interactivity. As a user, you can easily explore our photos and videos on a map without having to leave the map. In addition, you can click on the link in the pop-up window to view the full album or video.

Best GIS Solutions for Travel Bloggers to Create Interactive Maps

As recently as 10 years ago, getting professional-quality interactive maps meant shelling out hundreds, if not thousands of dollars every year in GIS software licensing fees. Even worse, there were very few online GIS programs to choose from back then.

Thankfully, that has all changed. Today, you there are an plenty of free online mapping programs available. Many of these programs are both incredibly powerful and easy to set up. While it’s certainly not a requirement, I highly recommend investing in a developer to install, connect, and integrate the maps on your website. They’ll be able to connect your maps directly to your database. As a result, new content will automatically be added to your maps, allowing them to effortlessly scale with your business. It’ll cost more up front, but with a good developer, you’ll save money in the long run.

We’d love to help you get set up with your maps. If you want to discuss your project further or get a free quote to add maps to your travel blog, please get in touch with us today. Or if you’re still just kicking tires, please feel free to browse our catalog of GIS products and services.

Google Maps or Bing Maps

If you’re just starting out or are looking for a simple solution, Google or Bing Maps are great options! Both platforms allow you to create high-quality maps and embed them on your website free of charge. And best of all, you don’t need a developer. Instead, you’ll just copy and paste a short block of code into your website or blog.

Unfortunately, the simplicity of both Google and Bing Maps leave them with some downsides. Most notably, if you have a dynamic dataset or high traffic volumes, you’ll run into issues. Neither platform is built to display large datasets. As a result, you’ll struggle to scale your application and will run into high API fees to generate the basemaps as your organization grows.

Mapbox

Mapbox is a direct competitor of both Google and Bing Maps, but it offers so much more. As a result, I typically recommend that my clients use Mapbox over Google or Bing. Mapbox gives you much finer control over your mapping application, allowing you to scale it both up and down as your business evolves. And best of all, it’s completely free to use, unless you have more than 50,000 map loads per month.

Mapbox map of Boston, in a light grey color scheme
Screenshot of a Simple Mapbox Map of Boston, Massachusetts

However, my favorite Mapbox feature is its use of vector tiles, which allows you to display huge amounts (read: gigabytes) of data that load and respond extremely fast.

  • Fully customizable maps
  • Much friendlier pricing than Google or Bing
  • Supports both proprietary and open source data formats
  • Offers a software development kit to add your own functionalities and customizations
  • Developer optional to use maps or embed them on your website

Leaflet

Leaflet is a simple, yet powerful open source JavaScript library that displays two-dimensional maps on your website or blog. Because it’s hosted on your server, you don’t need to worry about API fees, regardless of how much or how little traffic you get. It’s lightweight, fast, powerful, and completely customizable. Furthermore, it has an extensive library of plugins and extensions if you need additional functionality. If you don’t know JavaScript, you’ll need a developer for the initial set up of your maps. You can easily connect Leaflet to nearly any type of database or data repository. As a result, Leaflet maps will easily scale with your business once it’s set up.

A popup box on a sample Leaflet interactive map
Screenshot of a Data Point on a Leaflet Map

Thankfully, Leaflet requires very little maintenance once you get it up and running. In fact, I tell my clients that it’s often more cost-effective to pay for Leaflet maintenance as you need it instead of paying a recurring monthly maintenance fee. Yes, there are obviously exceptions to that rule. However, for the vast majority of people, Leaflet is an extremely cost-effective way to add high-quality maps to your website or blog.

Cesium

Like Leaflet, Cesium is an open source JavaScript library that creates powerful 3D maps to “unleash the power of 3D data”. Their maps are engineered to maximize performance, precision, and experience. With Cesium, it often feels like there is no limit when it comes to what you can do with a map. In fact, Cesium also includes a timeline, so you could make the argument that its maps are four dimensional instead of three.

Furthermore, they’ve even created their own 3D vector tile format that lets you load enormous datasets in seconds. For example, check out Cesium’s demo that loads 3D models of nearly every building in New York City. It’s fast, fluid, and responsive. For additional demos, have a look at Cesium’s Use Cases. You’ll find examples from many different industries, applications, and regions.

Screenshot of a cockpit from a flight simulator built on Cesium's 3D mapping technology
Someone even built a flight simulator on the Cesium platform

You can get an incredible amount of power and functionality out of Cesium’s free base functionality. For the average travel blog, the free functionality is probably more than you need. However, if you want to harness its full potential, you should at least look into some of the paid add-ons for Cesium. Those paid add-ons will streamline your workflow and optimize your data. As a result, your users will ultimately have a better experience.

Terria

If you’re trying to decide between Leaflet or Cesium, why not use both? Originally developed as an open source platform for the Australian Government, Terria lets your users choose whether they want to view a two or three dimensional map. And you can probably see where this is going. Leaflet powers Terria’s two dimensional maps, which Cesium is behind its three dimensional maps.

The best feature of Terria, however, is its user interface. Easily organize and browse through a huge number of datasets. It uses Cesium’s 3D data optimization to ensure your map remains fast and responsive, even if your datasets are massive. Use Terria’s slider to compare datasets side-by-side. It even includes a feature for you to build stories with your data and share them with your audience.

I use Terria for all of my mapping needs, and also recommend it for most of my clients. Its power and responsiveness across all devices, including mobile phones, coupled with its flexibility and minimal programming required to set it up make it the optimal platform for me. My users and clients have never complained about being confused using Terria, and are often impressed at how easy it is to analyze huge amounts of data. And best of all, I can set it up so it scales up and down as I need without needing hardly any maintenance.

If you want your travel blog to stand out from the rest, adding fully interactive maps with Terria is one of the easiest and most cost-effective way to do so. To learn more or get started, please get in touch with us or browse our online resources.

ESRI ArcGIS Online

If you have a complex dataset, but would prefer not to hire a developer, ESRI’s ArcGIS Online may be the best solution for you. Yes, it does have licensing fees, but you’ll get much of the functionality of the other applications we’ve discussed without needing a developer to set them up for you. Like the other platforms, ArcGIS online can easily handle large numbers of complex datasets, and plot those data on maps that need just a copy and paste to embed in your website. Plus ESRI is widely considered to be the industry standard for anything related to GIS and maps. If anything goes wrong for you, they have excellent documentation and support.

If you’re looking for a real-world example of ArcGIS Online in action, you’ve probably seen them already. Since the COVID-19 pandemic began, most dashboards that display maps of COVID-19 data use ArcGIS online.

Interactive map on Johns Hopkins University's COVID-19 dashboard
ESRI ArcGIS Online powers many COVID-19 Dashboards, including the one by John’s Hopkins University

Summary

PlatformFreeAPI FeesDimensionsDeveloperDynamic Data
Google/Bing MapsYesOptional2D OnlyNot RequiredNo
MapboxYes> 50K loads/mo2D OnlyOptionalYes
LeafletYesNot Required2D OnlyRequiredYes
CesiumYesNot Required3D OnlyRequiredYes
TerriaYesNot Required2D & 3DRequiredYes
ArcGIS OnlineNoN/A2D OnlyNot RequiredYes

Travel Bloggers, Take Your Interactive Maps to the Next Level

Once you have your new interactive map set up on your website or blog, simply adding a link to it is not enough. In addition, you should strategically embed them on different pages of your website to give your users the most immersive experience. For example, you’ll find the Matt Gove Photo Visual Media map embedded both on the home page of this blog and the main photo gallery page on the Matt Gove Photo website, in addition to being embedded above. I’ll continue to add maps as we go forward, too.

Interactive map embedded on the Matt Gove Photo website
You’ll find the Matt Gove Photo Visual Media Map strategically embedded across our websites

To figure out where to embed your interactive maps, have a look at your website’s analytics. Are there pages that have a lot of page views? Is there a specific page people are navigating to that the map could benefit? Are your visitors getting confused and leaving your website instead of navigating somewhere the map could help? Is there a logical place for your maps in your navigation or sales funnel?

Finally, as a travel blogger, you shouldn’t plot just your blog posts on the interactive map. Geotag photo albums, videos, social media posts, guides, fun activities, scenic drives, and much more. Don’t be afraid to make multiple maps, either. Try to use a platform like Terria or ArcGIS Online that organizes your datasets in a logical manner and makes it easy to both add and remove data from the map. If that’s not an option, don’t overwhelm your users with too much data on a single map. That’s one of the best ways to drive visitors off of your website and directly into the arms of your competitors.

Conclusion

Fast, professional quality interactive maps are one of the best ways travel bloggers can stand out from the crowd. Interactive maps are easy and cost-effective to implement and maintain. They’re also incredibly effective at retaining your visitors’ engagement and keeping them on your website. It boggles my mind why so many travel bloggers haven’t taken full advantage of the incredible potential interactive maps present to both grow your audience and keep your existing followers coming back for more.

Are you ready to take the next steps with interactive maps and bring your website or travel blog to the next level? As avid travelers and data science experts who specialize in online GIS and mapping applications, we’d love to help you take that next step in your journey. I invite you to please browse our catalog of GIS and mapping services. Then, get in touch or book a free info session with us to discuss your specific project. We can’t wait to hear from you.

Top Photo: Chapman’s Peak Drive on the Matt Gove Photo Scenic Drives Map
Cape Town, Western Cape, South Africa

The post Travel Bloggers: How to Stand Out with Powerful Interactive Maps appeared first on Matthew Gove Blog.

]]>
How to Boost Your GIS Productivity with Python Automation in 5 Minutes https://blog.matthewgove.com/2021/11/05/how-to-boost-your-gis-productivity-with-python-automation-in-5-minutes/ Fri, 05 Nov 2021 16:00:00 +0000 https://blog.matthewgove.com/?p=3385 Python Automation is one of the most powerful ways to improve your GIS workflow. In the past, many tasks in traditional GIS applications have had minimal support for writing your own code, and often required crude hacks to install obscure libraries. As Python has rapidly grown in both functionality and […]

The post How to Boost Your GIS Productivity with Python Automation in 5 Minutes appeared first on Matthew Gove Blog.

]]>
Python Automation is one of the most powerful ways to improve your GIS workflow. In the past, many tasks in traditional GIS applications have had minimal support for writing your own code, and often required crude hacks to install obscure libraries.

As Python has rapidly grown in both functionality and popularity, it is now widely supported across, and even built into many GIS platforms. Adding Python scripting to your GIS workflow can accomplish tedious hours-long tasks in seconds. Full automation of your GIS processes with Python will free you up to focus the more important aspects of your project, regardless of what industry you’re in.

Automate Your Desktop GIS Application

Did you know that ESRI ArcGIS and QGIS are both written in Python? As a result, Python automation integrates effortlessly with both GIS platforms. The Python libraries for each platform are incredibly powerful, fast, and easy to use.

However, be aware that the Python libraries for ArcGIS and QGIS are specific to each platform. If you ever decide to change platforms, you’ll need to rewrite all of your Python scripts.

QGIS Window with Python Console

I recommend starting small to get your feet wet with GIS Python automation. Start by automating the symbology and color of your data before diving into data manipulations, calculations, and analysis. Then you can start tackling more complicated processes, such as file conversions, modifying layers, switching projections, and much more.

Automate Your Web-Based GIS Application

Automating web-based GIS applications with Python is not quite as seamless as with ArcGIS or QGIS. However, you can easily argue that it’s even more powerful. Web-based GIS applications are a bit more complicated than desktop-based platforms. In addition to the GIS software, you often need special servers and databases that are designed specifically for geospatial data.

Thankfully, this increased complexity also means that there are more opportunities for automation. I use Python automation on nearly all of my web-based GIS applications. I don’t have tutorials for all of these yet, but here are a few of my favorites.

Python Automation Updates Our COVID-19 Dashboard Every Day

Remote Sensing Automation with Python

Most sensors these days come with Python libraries when you buy them. You should absolutely take advantage of those libraries. With Python, you can calibrate and configure the sensors exactly how you want them, not the way the manufacturer wants them.

In May of 2019, I installed sensors on the weather station I have at my house. The weather station runs on a network of Raspberry Pi’s. A Python script reads the data from each sensor, QA/QC’s it, and records it in the weather station’s database. If a sensor goes offline or makes a bad reading, the weather station pulls the data from the National Weather Service.

  • DIY Weather Station: Building a Solar Radiation Shield from Scratch
  • Wiring Power and Internet to the Sensors
  • Installing the Data Logger and Connecting the Sensors
  • Database Configuration and Programming the Sensor Readings
  • Troubleshooting a Sensor Gone Awry
https://youtube.com/watch?v=twZNWximYd0

Take your remote sensing automations even further. Use Python GeoPandas to plot your data on a map. Perform a high-level data analysis using pandas or matplotlib. You can easily automate the whole process or leave yourself as much manual control as you wish.

Data Entry Automation with Python

Without data, you don’t have a GIS application. You just have a map. Furthermore, geodatabases and data repositories come in all different shapes and sizes. Thankfully, Python can easily handle all of these data types and schemas thanks to its robust and dynamic data science libraries.

Python’s pandas library is one of the most powerful data analysis libraries available in any programming language. The fact that it’s free and open source is even more incredible, given how expensive licenses to proprietary software can be. pandas can handle just about any data format and size you throw at it.

However, pandas on its own does not support any geographical or location-based data. Enter Python’s GeoPandas extension of the pandas library. GeoPandas gives you the ability to analyze geospatial data and generate maps using the same tools you have in pandas. Easily populate a geodatabase or assemble a repository of any supported GIS format, including shapefiles, CSV, GeoJSON, KMZ, and much more. For more information, please visit our collection of GeoPandas tutorials.

Python GeoPandas can create beautiful maps without a GIS application

Data Analysis Automation with Python

With over 12 years of experience in professional data analysis, I know firsthand how tedious having repetitive tasks can be. Instead of the monotony of having to repeat those tasks over and over, why not automate them with Python? After all, Python developers created both pandas and matplotlib for that exact purpose. In the context of GIS, you can fully or partially automate many common tasks.

  • Repetitive tasks to prepare and/or format the data for analysis
  • Create maps of different areas using the same data parameters
  • Generate multiple maps of the same areas using different data parameters
Python has plenty of powerful data analysis libraries available for geospatial data

How to Trigger Your GIS Automation

To reach the nirvana of full automation, a Python script alone is not enough. You also need to automate the automation. Fear not, though, triggering your automation is the easy part. You have two options to choose from.

Trigger Your Automation to Run at a Set Time

The majority of GIS automations run at the same time every day. Our COVID-19 dashboard is the perfect example of this. We have a Python script that downloads the COVID-19 data from the Johns Hopkins University GitHub repository, parses it, and adds it to our database. Unfortunately, our web hosting plan does not allow us to fully automate the script, so we automate it on a local machine and then upload the database to the production server.

Scheduling the automation on your computer or server is quick and easy. On Linux and Mac OS, use the crontab command to schedule a cron job, which is a job that runs at fixed dates, times, and/or intervals. Alternatively, use the Task Scheduler on Windows. Both schedulers give you full flexibility to schedule jobs exactly when you want them to run.

Trigger the Script to Run When a Specific Event Occurs

Alternatively, not all jobs run at a specific time or interval. Have a look at the map of the Matt Gove Photo albums and videos. There is no logical need to run the job at a set time or interval. Instead, we update the map whenever we add a photo album or video to the database. As soon as the new album or video lands in the database, it automatically adds the data to the map.

In Python, the simplest way to trigger your GIS automation is a call to a function that runs the automation. For example, let’s look at the logic of adding photos and videos to the Matt Gove Photo map. In its simplest form, the logic for adding a photo album would look something like this.

# Add a Photo Album to the Database
add_photo_album_to_database(album_parameters)

# Once the database is updated, update the map
update_map(album_parameters)

This example is very oversimplified, but you get the point. For even finer control, use conditional logic and loops for precisely triggering your scripts exactly when you want.

Don’t Forget to Test Your Automation Scripts Before Putting Them into Production

We all make this mistake at one point or another. You beam with pride when you finish your automation script, and schedule it to run for the first time overnight. The next morning, you log in eagerly expecting to see the output of your automation. Instead, you see nothing, or even worse, you see an error message. You facepalm yourself because you forgot to test everything!

The best way to test your automation is to write a few quick unit tests once you finish your script. If you’re unfamiliar with a unit test, it tests individual units of code in your script. You tell the test the expected outcome for a given parameter, and then run that parameter through the block of code. If the script output matches the expected output, the test passes. If not, it fails.

For example, let’s say you programmed a calculator application. To set up a unit test for addition, execute 2 + 2 with the calculator, and see if you get 4. Repeat the process with unit tests for subtraction, multiplication, and division. The best part about unit tests is that you can run a lot of them in a short amount of time. If you’ve written them correctly, they’ll tell you exactly where in the script any problems are.

Use Creativity and Innovation in Your Python Automation

Once you get your feet wet with GIS automation using Python, keep automating. I encourage you to get creative and come up with new, innovative ways that will improve your workflow even further. The sky really is the limit when it comes to automation.

Conclusion

The days of managing bloated and complicated workflows with expensive software are a thing of the past. Python automation is the future, not just in GIS, but also in nearly every industry out there. Start out with simple tasks to whet your palette. Once you get a taste of it, don’t be afraid to tickle that creative or innovative itch we all have. You’ll be amazed at the amount of time and money it can save. Let us help you get started today.

Top Photo: View of Death Valley from Sea Level
Death Valley National Park, California – February, 2020

The post How to Boost Your GIS Productivity with Python Automation in 5 Minutes appeared first on Matthew Gove Blog.

]]>
How to Automate Region Mapping in TerriaJS with 49 Lines of Python https://blog.matthewgove.com/2021/10/15/how-to-automate-region-mapping-in-terriajs-with-49-lines-of-python/ https://blog.matthewgove.com/2021/10/15/how-to-automate-region-mapping-in-terriajs-with-49-lines-of-python/#comments Fri, 15 Oct 2021 16:00:00 +0000 https://blog.matthewgove.com/?p=3334 A little over a month ago, we examined the benefits of using region mapping in your TerriaJS applications. Region mapping allows you to reduce your GIS application’s data usage by over 99%, permitting you to display massive datasets on two and three-dimensional maps that load quickly and are highly responsive. […]

The post How to Automate Region Mapping in TerriaJS with 49 Lines of Python appeared first on Matthew Gove Blog.

]]>
A little over a month ago, we examined the benefits of using region mapping in your TerriaJS applications. Region mapping allows you to reduce your GIS application’s data usage by over 99%, permitting you to display massive datasets on two and three-dimensional maps that load quickly and are highly responsive.

Example region mapping in TerriaJS
Our COVID-19 Dashboard Map Makes Extensive Use of Region Mapping in TerriaJS

Despite the power and benefits region mapping offers, setting it up can be tedious, time consuming, and rife with type-o’s if you attempt to do it manually. This can be particularly frustrating if you have a large number of regions you want to map. Thankfully, there’s a much easier way. With just 49 lines of Python code, you can generate region maps of any size in just seconds, freeing up valuable time for you to focus on more important tasks.

What You’ll Need

You’ll need a few items to get started with your TerriaJS region mapping automation.

  • The ESRI Shapefile or GeoJSON file that you used to generate your Mapbox Vector Tiles (MVTs) for the region mapping
  • The Python script you’ll write in this tutorial
  • A Terminal or Command Prompt

What is a Region Mapping File

In TerriaJS, a region map consists of two JSON files.

  1. The actual region map
  2. The region mapping configuration file

The Python script we’re writing in this tutorial generates the actual region map. The region map tells TerriaJS the order that each polygon appears in the vector tiles. The configuration file instructs TerriaJS which vector tile parameters contain the region’s unique identifier, name, and more. You can easily write a short Python script to generate the configuration. However, unless you have a lot of region maps you’re generating, I find that the configuration is so short, it’s easier to do manually.

Convert all Shapefiles to GeoJSON Before Automating Region Mapping

If you have ESRI Shapefiles, convert them to GeoJSON before automating your region mapping. We do this for two reasons.

  1. Python can read and parse GeoJSON files natively. To parse shapefiles, you’ll need a third-party library such as GeoPandas.
  2. Mapbox’s Tippecanoe program, which generates the Mapbox Vector Tiles we use for region mapping, requires files to be input in GeoJSON format.

The GeoJSON should contain at least three properties for each feature. We covered these properties in detail in our previous article about region mapping, so I won’t repeat them here. You’re welcome to add as many properties as you want, but it should at the bare minimum contain the following three features.

  • A Feature ID, or FID.
  • A unique identifier you’ll use to identify the feature in the CSV data files you load into Terria.
  • The name of the feature

Most of our map tiles actually contain several unique identifiers that can be used. For example, countries actually have two sets of ISO (International Organization for Standardization) standards, plus a United Nations code that can be used to uniquely identify them. We can use any of these three, plus our own unique identifier to map them to our Mapbox Vector Tiles.

CountryISO Alpha 2 CodeISO Alpha 3 CodeUN Numeric Code
AustraliaAUAUS036
BrazilBRBRA076
CanadaCACAN124
FranceFRFRA250
MexicoMXMEX484
New ZealandNZNZL554
United StatesUSUSA840
Unique Country ID Examples

All right, let’s dive into the Python code.

First, Input Python’s Built-In json Library

The real magic of region mapping in TerriaJS is that all of the files are just JSON files, which are compatible with every popular programming language today. As a result, all we need to do the automation is Python’s json library. The json library comes standard with every installation of Python. We just need to import it. We will also be using Python’s os library to ensure the correct paths exist to output our region mapping files.

import json
import os

First, Write a Python Function to Extract the Feature ID from each Feature in the GeoJSON

This is because you need to go down several layers into the GeoJSON and we need this functionality a lot. It’s best practice to avoid string literals in programming, so we’ll use a function to extract the Feature ID from any given feature in the GeoJSON file.

def fid(f):
    return f["properties"]["FID"]

Define Your Input Parameters

We define the GeoJSON file name, the layer name in the TerriaJS region mapping configuration (regionMapping.json), and the property we’ll use as the unique identifier in our region mapping. Even if your MVT tiles have multiple unique identifier, you may only choose one for each region map. You’ll need to generate a second region map if you want to use multiple unique identifiers.

GEOJSON = "world-countries.geojson"
LAYER = "WORLD_COUNTRIES"
MAPPING_ID_PROPERTY = "alpha3code"
OUTPUT_FOLDER = "regionIds"

If you prefer not to have to manually edit the Python script every time you want to change files, you can easily update the Python script to extract this information from the filename or receive it through command line arguments.

Define the Output Path for Your Region Map File

I like to do this right away because it uses the input parameters. This both gets it out of the way so we don’t need to deal with it later and eliminates the need to scroll up and down looking for variable names if we were to define these at the end. We’ll structure our output paths so they can just be copied and pasted into Terria when we’re done. No renaming or reshuffling of files required.

geojson_filename = GEOJSON.replace(".geojson", "")
output_json_filename = "region_map-{}.json".format(geojson_filename)
output_fpath = "{}/{}".format(OUTPUT_FOLDER, output_json_filename)

In this example, the output filepath or our region mapping file will be regionIds/region_map-world-countries.json.

Read the GeoJSON into Python

Because GeoJSONs are just a specific type of JSON file, we can use Python’s built in json module to read and parse the GeoJSON file.

with open(GEOJSON, "r") as infile:
    raw_json = json.load(infile)

Sort the GeoJSON features by Feature ID

When you convert the GeoJSON to MVT tiles, Tippecanoe sorts the GeoJSON by Feature ID. For the region mapping to work correctly, the feature in the region map must appear in the exact same order as they appear in the Mapbox Vector Tiles. If they’re not in the exact same order, your data will be mapped to the wrong features in TerriaJS. What’s particularly insidious about this issue, is that it will appear as if your data is mapped to the correct county, even though it’s not.

When I first tried to set up region mapping for the World Countries vector tiles, I did not have the features in the right order. Data for the United States was mapped to Ukraine and labeled as Ukraine’s data. France’s data showed up in Fiji, and New Zealand’s data appeared to be Nepal’s. And that’s just to name a few. Except for a few countries at the top of the alphabet that start with “A”, this issue plagued every country. Once I sorted the GeoJSON features by Feature ID, the problem magically went away.

raw_features = raw_json["features"]
features = sorted(raw_features, key=fid)

Note here that we use the fid function we defined above to identify the Feature ID in the sorted() function.

TerriaJS Region Maps are just Arrays of Sorted Feature ID’s

To generate the region map for TerriaJS, all we need to do is just loop through the sorted features and create an array of the unique ID for each feature. Remember that the unique ID is different from the Feature ID. For world countries, TerriaJS comes with the Alpha 2 Codes built into it. For this tutorial, we’ll use the ISO Alpha 3 code as our unique identifier, which we defined in the MAPPING_ID_PROPERTY variable in the use input. If you’ve forgotten, the alpha 3 code is just a 3-letter code that identifies each country. For example, use “USA” for the United States, “CAN” for Canada, “JPN” for Japan, and so forth.

We’ll generate the region map with a simple for loop.

region_mapping_ids = []

for f in features:
    properties = f["properties"]
    region_map_id = properties[MAPPING_ID_PROPERTY]
    region_mapping_ids.append(region_map_id)

Assemble the full TerriaJS Region Mapping JSON in a Python Dictionary

TerrisJS region mapping files are breathtakingly simple. They require three parameters.

  • layer: The name of the layer in the TerriaJS Region Mapping configuration (regionMapping.json). We defined it above with the LAYER variable in the user input.
  • property: The name of the unique identifier in your vector tiles that you’ll use in the CSV files you load into Terria. For the world countries, we’re using the alpha2code identifier. We defined this in the user input using the MAPPING_ID_PROPERTY variable.
  • values: This is the the array of sorted feature ID’s we created in the previous section

It’s important to note that Python does not actually support JSON format. Instead, when you read a JSON file into Python, it actually converts the JSON into a Python dictionary. When we export the dictionary to JSON format, Python just simply does the conversion in the opposite direction.

Anyway, the TerriaJS region mapping JSON should look like this as a Python dictionary.

output_json = {
    "layer": LAYER,
    "property": MAPPING_ID_PROPERTY,
    "values": region_mapping_ids,
}

Also, don’t forget to create the output directory if it doesn’t exist. Your Python script will crash without it.

if not os.path.isdir(OUTPUT_FOLDER):
    os.makedirs(OUTPUT_FOLDER)

Finally, Write the TerriaJS Region Mapping JSON to a .json File

Thanks to Python’s json module, we can output the region mapping file with just a couple lines of code.

with open(output_fpath, "w") as ofile:
    json.dump(output_json, ofile, indent=4)

If you did everything correctly, the region mapping JSON you just created should look like this.

{
    "layer": "WORLD_COUNTRIES",
    "property": "alpha3code",
    "values": [
        "AFG",
        "ALB",
        "DZA",
        "AND",
        "AGO",
        "ATG",
        ...
    ]
}

But we’re not quite done, yet!

Add Your New TerriaJS Region Mapping to the Configuration File

The final step is to add your new region map to the TerriaJS Region Mapping configuration file. By default, the configuration file is located at data/regionMapping.json. However, if you can’t find it, it’s defined in the config.json file at the root of the Terria application. You are more than welcome to automate this, too, but I find that it’s so simple, it’s often easier to just do manually.

The configuration file instructs TerriaJS how to interpret each region map. You’ll need to include several parameters.

ParameterDescription
layerNameThe name of the layer in the Mapbox Vector Tiles. If you can’t remember it, check your vector tiles’ metadata.
serverThe URL from where your Mapbox Vector Tiles are served
serverTypeThe type of files the server is serving. For this tutorial, use MVT, which stands for “Mapbox Vector Tiles”.
serverMinZoomThe minimum zoom level of the vector tiles
serverMaxNativeZoomThe maximum zoom level the vector tiles support natively
serverMaxZoomThe maximum zoom level the vector tiles support
regionIdsFileThe path to the TerriaJS Region Mapping JSON we created in this tutorial
regionPropThe name of the property in the vector tiles that contains the unique identifier we’re using in the region map.
aliasesAny column headers in the CSV data file that TerriaJS should interpret as the unique identifier of your region map
descriptionA description of the region map
bboxThe bounding box of your vector tiles, in the format [west, south, east, north]
namePropThe name of the property in the vector tiles that contains the name of each feature

Using the layer we defined in the LAYER variable as the JSON key, your regionMapping.json configuration file should look like the following.

{
    "regionWmsMap": {
        ...
        "WORLD_COUNTRIES": {
            "layerName": "world-countries",
            "server": "https://yourvectortileserver.com/worldcountries/{z}/{x}/{y}.pbf,
            "serverType": "MVT",
            "serverMinZoom": 0,
            "serverMaxNativeZoom": 3,
            "serverMaxZoom": 8,
            "regionIdsFile": "data/regionIds/region_map-world-countries.json",
            "regionProp": "alpha3code",
            "aliases": ["alpha3code", "country_id", "COUNTRY_ID"],
            "description": "World Countries",
            "bbox": [
                -179.99999999999999999999999,
                -85,
                179.999999999999999999999999,
                85
            ],
            "nameProp": "name"
        },
        ...
    }
}

Create a Dummy CSV File to Test it Out

The funnest part of any project is seeing it all come to life once you’re done. To do this, create a dummy CSV file to test that your region mapping actually works. Pick a bunch of countries at random, assign them some random values as a dummy parameter, and see if they show up on the map.

alpha3codeValue
USA7
CAN4
DEU15
THA12
AUS9
BRA3

To load the file into TerriaJS, just click on the upload button at the top of the workbench, next to the “Browse Data” or “Explore Map Data” button. If the region mapping is working properly, you should see your data appear appear on the map.

Our dummy CSV data displayed on a choropleth world map
The Dummy CSV Data on a World Map

Conclusion

Region mapping is one of the most powerful and efficient ways to display enormous amounts of data in TerriaJS. Automating the region mapping process only saves you valuable time and makes your application even more powerful.

While manually region mapping feature sets such as the 50 US states or the roughly 200 world countries may initially seem manageable, it rapidly becomes a nightmare once you try to scale it up. Sticking just within the United States, what if instead of the 50 states, you were mapping America’s more than 3,000 counties? Our COVID-19 Dashboard map does just that. Or even worse, the United States has over 41,000 postal codes and more than 73,000 census tracts. Can you imagine having to assemble those region maps manually, or the opportunity for type-o’s manually entering tens of thousands of data points?

Instead, save yourself the time, money, and hassle. We’ve made the Python script available for free on Bitbucket so you can configure region maps of any size in TerriaJS in just seconds. And if you ever run into issues, we’re always here to help you with any questions you may have. Happy mapping!

Top Photo: Snow-Capped Sierra Nevada Provide a Spectacular Backdrop for Lake Tahoe’s Brilliant Turquoise Waters
Meeks Bay, California – February, 2020

The post How to Automate Region Mapping in TerriaJS with 49 Lines of Python appeared first on Matthew Gove Blog.

]]>
https://blog.matthewgove.com/2021/10/15/how-to-automate-region-mapping-in-terriajs-with-49-lines-of-python/feed/ 1
How to Set Up and Automate Region Mapping in Terria https://blog.matthewgove.com/2021/09/10/how-to-set-up-and-automate-region-mapping-in-terria/ Fri, 10 Sep 2021 16:00:00 +0000 https://blog.matthewgove.com/?p=3222 In today’s media-rich internet, load times are critical to your web application’s success. There is no industry for which that rings true more than the online mapping and GIS community. In the past, even small to medium-sized datasets were notorious for crippling web-based GIS applications and crashing web browsers. Thankfully, […]

The post How to Set Up and Automate Region Mapping in Terria appeared first on Matthew Gove Blog.

]]>
In today’s media-rich internet, load times are critical to your web application’s success. There is no industry for which that rings true more than the online mapping and GIS community. In the past, even small to medium-sized datasets were notorious for crippling web-based GIS applications and crashing web browsers. Thankfully, the leaders of the online GIS community have come up with a solution that I used to reduce data usage in my COVID-19 Map by 99.92%. And that’s just for one country. Today, you’re going to learn how to achieve similar results by setting up region mapping in Terria.

What is Region Mapping?

Region mapping is the process of using unique identifiers to map a dataset to a separate map geometry. While this geometry can be any polygon, line, or point on a map, it is most often some kind of physical or political boundary, ranging from state and country borders down to census tracts, neighborhoods, and property lines. The benefits of region mapping are profound for web-based GIS applications such as Terria and Cesium.

  • Load map geometries once in region mapped datasets vs every data point in datasets that do not use region mapping
  • In Terria, your dataset is stored in CSV format, which can opened with everyday tools such as Microsoft Excel
  • Vector tiles used in region mapping are typically between 1% and 10% the size of most other GIS formats, including Shapefiles, GeoJSON, and KML/KMZ.
  • Reuse vector tilesets over multiple datasets to easily scale your application up and down. The geometries I use to track COVID-19 can be reused to track weather, elections, Census data, population, and more.
  • Region mapping only loads the visible part of the map geometry in Terria. Without vector tiles, you must load the entire geometry, regardless of whether or not it’s visible.

Region Mapping Performance in Terria

So just how much of a performance boost do you get using region mapping? Let’s look at the actual performance of loading the geographies of the Canadian provinces that we use in our COVID-19 map. Keep in mind that these numbers are for loading the geometry only. They do not load any COVID-19 data.

ParameterGeoJSONRegion Mapping/Vector Tile
Entire Geometry Size141 MB26 MB
Whole Country Overview on Map141 MB643 kB
Whole Country Overview Load Time (100 mbit/sec)11.28 sec0.05 sec
Provincial Overview: Ontario141 MB281 kB
Ontario Overview Load Time (100 mbit/sec)11.28 sec0.02 sec

As you can see, the region mapping wins hands down. This effect only gets compounded once you add the COVID-19 data. If you’re not using region mapping, the map geometry needs to be loaded for every data point you have. When you take that 141 MB GeoJSON and multiply it by 550-plus days of the pandemic, you’re looking at about 70 GB of data for just the geometry alone! And that number gets bigger with each day that passes. With region mapping, you only load the 643 kB of geometry to see the overview of Canada, regardless of how many days the pandemic drags on for.

If it weren’t for region mapping, we would not be able to publish our COVID-19 maps because the large datasets would crash your browser.

I do want to point out that GeoJSON currently does not support time-series data. However, if you convert it to a format that does, such as CZML, you’re still looking at similar sizes and load times for the geometry.

What You’ll Need to Set Up Region Mapping in Terria

The Terria developers have thankfully made it very easy to set up region mapping. You’ll need a few tools to do it.

  • Your map geometry, in either GeoJSON format or a format that can be easily converted to GeoJSON, such as an Esri Shapefile.
  • A means to convert your geometry to GeoJSON. I recommend either a GIS program or Python GeoPandas.
  • Your favorite text editor
  • [Optional] Python, if you want to automate the process. We’ll cover automating this process in detail in a future tutorial.

Step 1: Configure Your GeoJSON or Shapefile

Before we convert our GeoJSON into vector tile format, we need to make sure each feature has the necessary parameters for region mapping. At the very minimum, each feature needs to have a feature ID and a unique ID that can easily identify it.

Set a Numeric ID for Each Feature in the GeoJSON

The feature ID is simply an incremental number starting at zero or one. We’ll use the feature ID to tell Terria the correct order that each feature appears in the vector tileset. Without this feature ID, your features will not display the correct dataset on the map unless they are in numerical or alphabetical order based on your ID. For example, let’s stick with the COVID-19 data and look at US States. Your GeoJSON is in alphabetical order by state name, and you’re using the 2-letter postal abbreviation as the unique ID.

Here’s where things go awry without a feature ID. If your GeoJSON is in alphabetical order by state name, Maine comes before Maryland. Likewise, Maryland comes before Massachusetts. However, using the postal abbreviations, that order changes. Massachusetts (MA) comes first, followed by Maryland (MD), and then Maine (ME). Because you specified the postal abbreviations as the unique ID for region mapping, Terria assumes that they are in alphabetical order by postal abbreviation. However, you have actually set them in alphabetical order by state name.

As a result, Terria will incorrectly determine that Maine’s data belongs to Massachsetts and vice-versa. If you clicked on Maine on the Map, the data table would be labeled as Maine, but it would actually display Massachusetts’ data. With a feature ID, you can actually tell Terria that the correct order of postal abbreviations is ME, MD, and then MA. We’ll circle back to this shortly.

Once you get your GeoJSON into the order that you want it to be in, go through an assign an incremental Feature ID to each feature in the GeoJSON. If you’re using the US States in alphabetical order by name, Alabama would be 1, Alaska would be 2, Arizona would be 3, and so on.

Include a Unique Identifier You’ll Use in Your Datasets

You’re more than welcome to use your Feature ID in the datasets. However, I really don’t recommend doing that because you won’t be able to remember them. Instead, assign them a standardized ID, such as an ISO, FIPS, or postal code. Let’s say you want to plot some data for the New England States. If you had to update the dataset several years down the road, which identifiers would be easier to recognize when you open the data file?

  • ME, NH, VT, MA, RI, CT
  • 21, 30, 46, 19, 39, 7

Interestingly, there’s still a problem using just the postal abbreviations as your unique identifier. What happens if you’re working across multiple countries? You’re bound to run into a conflict at some point where two countries use the same abbreviation. Take Massachusetts as an example. There are 23 countries that use “MA” as a state or provincial abbreviation.

To avoid those conflicts, simply use the ISO 3166-2 codes to identify each state. The ISO codes are standardized worldwide and prepend the 2-letter country abbreviation to the state or province abbreviation to make it a unique identifier worldwide. Even if you only plan to operate in just one country, you should still use the ISO codes. You never know when you may need to expand.

ISO 3166-2 CodeState or ProvinceCountry
US-MAMassachusettsUnited States
ES-MAMálagaSpain
BR-MAMaranhãoBrazil
CD-MAManiemaDem. Republic of Congo
JO-MAAl MafraqJordan

Our Recommended GeoJSON Configuration

Our region mapping files consist of four properties.

  • Feature ID
  • ISO Code or other standardized unique identifier
  • Unique numerical identifier that’s tied to our master geodatabase
  • The feature or entity’s name

We include two unique identifiers because it gives us flexibility in how we structure our datasets. While we primarily use the standardized identifier, there are certainly instances where it’s more beneficial to use the once that’s directly linked to our master geodatabase. If you aren’t using a master database, by all means omit that property.

When you put it all together, here’s an example of what the properties for Massachusetts look like in our GeoJSON.

{
    "FID": 19,
    "iso3166_2": "US-MA",
    "state_id": 4239,
    "name": "Massachusetts"
}

Step 2: Convert the GeoJSON to Mapbox Vector Tile Format

To convert the GeoJSON file to vector tile format so you can use it for region mapping in Terria, you’ll need to use a tool called Tippecanoe. Mapbox created Tippecanoe specifically for converting GeoJSON files to vector tiles, but it comes with a catch. It’s Unix based, so it only runs on Mac OS or Linux. It does not work on Windows. You can install Tippecanoe for free from GitHub.

Once you have Tippecanoe installed, it only takes one command from your command line to generate the vector tiles. We use the following command to generate vector tiles of state and provincial boundaries:

tippecanoe -z10 -e states --drop-densest-as-needed mexico-states.geojson -l mxstates

So what does this all mean? Let’s break it down. Please consult the Tippecanoe documentation or reach out to us for further details. We strongly recommend you standardize your naming conventions across all vector tile properties so they’re consistent and easy to reference in the future.

CommandMeaning
tippecanoeUse the Tippecanoe program
-z10Include zoom levels from 0 to 10 in the vector tile set
-e statesOutput raw vector tile files to a folder called “states”
--drop-densest-as-neededDrop the least visible features at each zoom level as needed to reduce file size
mexico-states.geojsonThe path to the GeoJSON we want to convert to vector tiles
-l mxstatesThe name of the layer you’ll reference in the region mapping in Terria

Upload Your Vector Tiles to a Mapping Server

Finally, you’ll want to upload your newly created vector tiles to a mapping server. However, be aware that they will not work “out-of-the-box” if you upload them to a standard web host. However, you can configure standard web hosts to serve vector tiles. We’ll cover setting up a mapping server in a future tutorial. In the meantime, if you want to just upload your vector tiles and be done with them, put them in an Amazon S3 bucket, which is configured to serve map tiles by default.

Step 3: Create Your Region Mapping File For Terria

In this step we will create the actual map that tells Terria which unique identifiers correspond to which geometries. The region map is stored in a JSON file. Creating the region map is very prone to type-o’s and human error, so I highly recommend automating this step with Python. We’ll cover automation in a future tutorial. Before we begin, you’ll need a few items from previous steps.

  • The name of the layer in your vector tiles from Step 2
  • Your unique identifier from Step 1 (not the Feature ID)
  • The order that each feature appears in the GeoJSON from Step 1

Remember the Feature ID we discussed in Step 1? If you followed our advice, you put the feature in the GeoJSON in numerical order by that Feature ID. To generate your region map, all you have to do is sort your GeoJSON by Feature ID and pull your unique identifiers out of the GeoJSON in that same order. It’s that simple.

Recall in Step 1 that we had a GeoJSON of US States sorted Alphabetically by state name. We used the ISO 3166-2 codes, which are based on postal abbreviations, as our unique identifier. To generate the region map, simply list the ISO 3166-2 codes for each state in alphabetical order by name. Here’s our region map showing the preserved alphabetical order of Maine, Maryland, and Massachusetts by name. I omitted the rest of the states to save space.

Example Using US State Geometries

{
    "layer": "usstates",
    "property": "iso3166_2",
    "values": [
        ...
        "US-KY",
        "US-LA",
        "US-ME",
        "US-MD",
        "US-MA",
        "US-MI",
        "US-MN",
        ...
    ]
}
ParameterDefinitionDefined In
layerLayer Name of the Vector Tileset, defined with the -l flag in TippecanoeStep 2
propertyThe key of your unique identifier you defined in the GeoJSONStep 1
valuesYour unique identifiers, in the exact order in which their features appear in the GeoJSONStep 1

If you haven’t already, create a folder called data in the root of your Terria application. Inside that folder, create another folder called regionIds. Save your region map in that regionIds folder. You can call it whatever you want, but I recommend naming it something that you can easily identify. For the US States name it something like region_map-us-states.json.

Step 4: Create or Update Your Terria Region Mapping Configuration

Now that we’ve created our region map, we need to tell Terria both to use it and how to use it. We’ll use a JSON file to provide Terria with our region mapping configuration. Terria actually comes with a default region mapping configuration for Australia, but I find it’s much easier if we just define our own instead of trying to add to Terria’s.

First, open the data folder you created in the root directory of Terria at the end of Step 3. Inside, create a file called regionMapping.json. This file will store the region mapping configuration for our Terria application. Open the regionMapping.json file in your favorite text editor. You’ll paste a key/value pair into it for each vector tileset you wish to use for region mapping. Set the keys so they’re easy to identify. In our COVID-19 map, we use keys such as US_STATES, US_COUNTIES, CA_PROVINCES, and WORLD_COUNTRIES.

Example Using US State Geometries

{
    "US_STATES": {
        "layerName": "usstates",
        "server": "https://yourmaptiledomain.com/us-states/{z}{x}{y}.pbf",
        "serverType": "MVT",
        "serverMinZoom": 0,
        "serverMaxNativeZoom": 3,
        "serverMaxZoom": 10,
        "regionIdsFile": "data/regionIds/region_map-us-states.json",
        "regionProp": "iso3166_2",
        "aliases": ["ISO3166_2", "US_STATE_ID"],
        "description": "US States",
        "bbox": [
            -179.999999999999,
            18,
            179.999999999999,
            72
        ],
        "nameProp": "name"
    }
}
ParameterDefinitionDefined In
layerNameLayer Name of the Vector Tileset, defined with the -l flag in TippecanoeStep 2
serverThe URL to your map tile serverStep 2
serverTypeSet to “MVT”, which stands for Mapbox Vector TilesN/A
serverMinZoomThe minimum zoom we defined when we created the vector tileset in TippecanoeStep 2
serverMaxNativeZoomThe maximum zoom level the server supports nativelyN/A
serverMaxZoomThe maximum zoom we defined when we created the vector tileset in TippecanoeStep 2
regionIdsFilePath to the region map JSON fileStep 3
regionPropThe key of your unique identifier that you defined in the GeoJSONStep 1
aliasesAdditional column headers in your dataset that Terria will recognize as your unique identifier. If you want the Terria’s drag and drop feature to work, one of these must be set to a unique value.Step 1
descriptionDescription of your region mapN/A
bboxBounding box that contains the map geometry, in degrees of latitude and longitude. It has the format [west, south, east, north].N/A
namePropThe key of the GeoJSON feature’s property that contains the feature’s nameStep 1

Step 5: Link Your CSV Data to the Region Map in the Terria Initialization File

The final step completes the link between your dataset and the vector tiles. We’ll update your initialization file to tell Terria which region map to use for each dataset in your catalog. You can find your initialization files in Terria’s init directory. All you have to do is add your CSV dataset to your initialization file. At the very minimum, you’ll need the following.

{
    "name": "United States: COVID-19 by State",
    "type": "csv",
    "url": "data/us-covid19-by-state.csv"
    "tableStyle": {
        "regionVariable": "iso3166_2",
        "regionType": "US_STATES",
        "dataVariable": "New Cases",
        "columns": Outside the Scope of this tutorial; Refer to Terria Documentation
    }
}
ParameterDefinitionDefined In
nameThe name under which your dataset will appear in Terria’s data catalogN/A
typeSet to “csv” because your data is in CSV formatN/A
urlThe URL where the CSV file containing your data is locatedN/A
tableStyle.regionVariableThe key of your unique identifier that you defined in the GeoJSONStep 1
tableStyle.regionTypeThe key of our region map in the Terria Region Mapping configurationStep 4
tableStyle.dataVariableThe header of the column that contains the data you want Terria to display when it initially loads your datasetN/A
tableStyle.columnsData formatting that’s outside the scope of this tutorial. Please consult the Terria documentation for details.N/A

If you’ve set everything up right, you’ll see each state get colored in when you use the ISO 3166-2 codes in your CSV file.

Example CSV Snippet of COVID-19 Data

ISO3166_2DateConfirmedDeadNew CasesNew Deaths
US-CA2020-07-01238,6816,1697,44987
US-CA2020-07-02246,5506,2657,86996
US-FL2020-07-01158,9973,5506,56345
US-FL2020-07-02169,1063,61710,10967
US-TX2020-07-01172,3862,5089,25550
US-TX2020-07-02179,1112,5476,72539
US-HI2020-07-019261890
US-HI2020-07-0194618200
US-NY2020-07-01394,07932,04362511
US-NY2020-07-02394,95432,06487521

Conclusion

Region mapping with vector tiles is the most powerful and efficient way to display massive datasets in web-based GIS platforms such as Terria and Cesium. Region mapping can seem daunting and even a little intimidating the first time you try it. Trust me, it took me a long time to figure it out, too, when I first started working with it.

If you’re struggling with region, please don’t hesitate to reach out to me personally or leave a comment below. The Terria developers are generally pretty responsive as well. Just a few small fractions of a second can be the difference between your mapping application taking off and failing. Are you willing to take that chance with slow, bloated file formats?

Top Photo: Region Mapping for the US, Canada, and Mexico on our COVID-19 Map
September, 2021

The post How to Set Up and Automate Region Mapping in Terria appeared first on Matthew Gove Blog.

]]>
Weird Geography: 10 Bizarre and Unusual Facts About Russia https://blog.matthewgove.com/2020/08/30/weird-geography-10-bizarre-and-unusual-facts-about-russia/ Mon, 31 Aug 2020 00:15:00 +0000 https://blog.matthewgove.com/?p=1556 It’s been a couple of months since the most recent edition of the Weird Geography series took us to France, and I don’t know about you, but I’m certainly ready to (virtually) get back on the road. It’s been a miserably hot summer here in Arizona, so I’m thinking we […]

The post Weird Geography: 10 Bizarre and Unusual Facts About Russia appeared first on Matthew Gove Blog.

]]>
It’s been a couple of months since the most recent edition of the Weird Geography series took us to France, and I don’t know about you, but I’m certainly ready to (virtually) get back on the road. It’s been a miserably hot summer here in Arizona, so I’m thinking we should head somewhere that’s well-known for being really cold. While France is known for having territories and overseas departments scattered all over the world, today we’ll be exploring the geography of the largest country in the world: Russia.

1. Russia spans 11 time zones and is permanently on Daylight Saving Time

Did I mention Russia is a big country? When Vladivostok is heading to work in the morning, Moscow is just sitting down to dinner. To put the time zones into perspective, Boston and Bangkok are 11 hours apart. When it’s 7 PM in Boston, it’s 6 AM in Bangkok.

Map of Russia on a globe

2. Russia shares a land border with 14 other countries.

Amazingly, it does not stand alone in bordering the most number of countries. China also shares a land border with 14 other countries. Russia’s longest land border is with Kazakhstan, which checks in at 6,847 km (4,254 mi), while Russia’s shortest international border is its 17 km (10.5 mi) border with North Korea.

And speaking of international borders…

3. You can travel over land from Finland to North Korea and only pass through a single country.

Despite Helsinki and Pyongyang sitting nearly 7,000 km (4,300 mi) apart as the crow flies and having a six-hour time difference, Russia is the only country that separates the two. However, this journey is impossible in reality since there are no legal border crossings between Russia and North Korea. Maybe I should have used China as the example instead.

Geography showing Russia between Finland and North Korea

4. At their closest points, Russia and the United States are only 4 km (2.5 mi) apart.

The Diomede Islands sit in the middle of the Bering Strait between Wales, Alaska and Naukan, Russia (Наукан). Big Diomede is a Russian island, while Little Diomede is part of Alaska. In a true geographical oddity, the two islands sit just under 4 km (2.5 mi) from each other, and due to the International Date Line running between them, there is a 21-hour time difference between Big and Little Diomede.

Map showing the Diomede Islands, which are the closest land points between Russia and the United States

Bonus Fact: The mainland-to-mainland distance between Wales, Alaska and Naukan, Russia is 80 km (50 mi). For perspective, the width of the Phoenix metro (measured from Goodyear to Apache Junction) is about 90 km (55 mi).

All right, enough about borders, let’s move on.

5. Russia is home to the coldest town on Earth

The Siberian town of Oymyakon (Оймякон) lays claim to being the coldest inhabited place on Earth. The town has very unique terrian and topography, as it sits in a river valley surrounded by mountains. The valley traps frigid air year-round, creating the extremely cold climate in town.

Not surprisingly, winters in Oymyakon are brutal, with average winter high temperatures hovering around -43°C (-45°F). Low temperatures between November and March routinely drop below -50°C (-58°F).

The second coldest temperature ever recorded on Earth was recorded at Oymyakon on 6 February, 1933, when the mercury dipped to -67.7°C (-89.9°F). Oymyakon’s weather station has been in operation since the 1920’s and has never recorded a temperature above freezing between 25 October and 17 March. Only Antarctica has recorded colder temperatures than Oymyakon.

Bonus Fact: On 28 July, 2010, Oymyakon measured its all-time record high of 34.6°C (94°F). When coupled with its all-time record low mentioned above, it has a temperature range of 102°C (184°F). The Russian towns of Verhoyansk, Yakutsk, and Delyankir, along with Fort Vermillion, Alberta, Canada are the only other places on earth that have observed a temperature range greater than 100°C.

6. Russia is home to the world’s largest and deepest lake, which contains over 20% of the Earth’s fresh water.

Lake Baikal (озеро Байкал) sits on the border between the Irkutsk Oblast and the Buryat Republic in southern Siberia. At 636 km (395 mi) long with a maximum depth of 1.64 km (5,387 ft), it contains more water than the Great Lakes combined, with a volume of 23,615 cubic km (5,670 cubic mi). Baikal is also considered to be the world’s oldest, cleanest, and clearest lake, and is home to thousands of species of plants and animals. You can often see the bottom in depths up to 30 to 40 meters (100 to 130 ft), especially in the wintertime.

The lake was declared a UNESCO World Heritage Site in 1996, and is a top Russian tourist destination year-round.

Map showing Lake Baikal in southeastern Russia

7. There is a part of Russia wedged between Poland and Lithuania

Formerly part of East Prussia and Germany, the Kaliningrad Oblast (Калининградская область) is an isolated exclave of the Russian Federation sitting between Poland and Lithuania. In January 1945, the Soviet Union invaded and annexed all of East Prussia except for the city of Königsberg. After the Nazis were defeated in 1945, the Germans agreed to cede the city to the Soviets.

The Kaliningrad Oblast is home to Russia's only ice-free port on the Baltic Sea

In 1946, the Soviets changed the name of Königsberg to Kaliningrad. By 1950, the remaining German population had been forcibly expelled, and Russian replaced German as the oblast’s official language. When the Soviet Union collapsed in 1991, the Kaliningrad oblast was designated part of Russia. Kaliningrad has become even more isolated since Poland and Lithuania joined the European Union and significantly tightened their border security.

Today, Kaliningrad is the only Russian port on the Baltic Sea that is ice-free year round, and plays a strategic role in the Russian economy and military.

8. Moscow is closer to both Churchill, Manitoba and Libreville, Gabon than Ozernovskiy, Russia.

The expansive geography of Russia has been known to yield some facts so absurd it seems there’s no possible way they can be true. Libreville, Gabon sits just 43 km (27 mi) north of the equator in central Africa, while Churchill, Manitoba is on the Hudson Bay in Canada about 140 km (87 mi) south of the provincial border with the Northwest Territories.

What’s even harder to believe? The distance from Churchill to Libreville is 10,750 km (6,680 mi). Ozernovskiy (Озерновский) is located on the southern tip of the Kamchatka Krai, about 1,100 km (700 mi) northeast of Japan.

Map showing Moscow, Ozernovskiy (Russia), Libreville (Gabon), and Churchill (Canada)

Here’s how the distances from Moscow break down:

CityDistance from
Moscow (km)
Distance from
Moscow (mi)
Churchill, Manitoba, Canada6,5844,091
Libreville, Gabon6,6594,138
Ozernovskiy, Russia6,8384,249

9. The US State of Alaska was once part of Russia

Have you ever heard someone say that Alaska stretches so far west it actually goes into the Eastern Hemisphere? Well, Russia once stretched so far east it went into the Western Hemisphere. Russia tried unsuccessfully to colonize Alaska, which was known as Russian America, before ultimately deciding keeping the territory was not profitable.

In 1867, the United States purchased Alaska from Russia for $7 million, in a deal known as Seward’s Folly. Ironically, Russia sold Alaska to the US because they saw Canada as a much bigger threat at the time. Some Russian town names, churches, and sayings remain preserved in Alaska to this day.

10. Russia is larger than the surface area of Pluto.

I can’t think of a scenario where this fact would be anywhere remotely close to useful, but it’s a cool statistic nonetheless, and would make for a really good trivia question. Pluto’s surface area is 16.7 million square kilometers, while Russia checks in at a whopping 17.1 million square kilometers.

Bonus Fact #1: The letters И and Я in the Cyrillic alphabet are actually vowels.

I’ve never quite understood why so many Americans think these are the Cyrillic letters for “N” and “R”. They’re just not. Both letters are actually vowels in all languages that use the Cyrillic Alphabet. The letter “И” is used basically the same way the letter “i” is used in English, while the letter “Я” makes a “ya” sound, which is common ending in many Russian words, including the Russian word for “Russia” – Россия – which is pronounced “Russ-ee-ya”.

Extra Bonus Fact: The Cyrillic alphabet is just the Greek alphabet with several extra letters to represent sounds in Slavic languages that are not used in Greek.

Bonus Fact #2: Bangladesh, which has 0.86% the land area of Russia, has a larger population.

Bangladesh, which covers roughly the same land area as the US State of Illinois, is one of the most densely populated countries in the world, while Russia, which has about twice the land area of Canada, is simply not.

CountryLand Area (sq km)2018 Population
Bangladesh148,460161.4 million
Russia17,098,242144.5 million

Well, that wraps up our geography adventure through Russia. Stay tuned for our next weird geography adventure, which will be a special spin-off of this rendition. Thanks for joining me, and I’ll see you next time. Спасибо. Увидимся в следующий раз.

Top Photo: An icy Vineyard Sound and Woods Hole Passage following a major blizzard
Woods Hole, Massachusetts – February, 2015

The post Weird Geography: 10 Bizarre and Unusual Facts About Russia appeared first on Matthew Gove Blog.

]]>