Efficient way to load many points on a web map

Sometime ago i had a project, that required to load many (10k+) points and show them on a map. Here i will show how you can do that

March 17, 2018 - 2 minute read -
web

Большое количество информации об этом на русском в моей книге https://getjump.me/posts/backends-book

Hey, sometime ago i had a project, that required to load many (10k+) points and show them on a map.

What you possibly now, that for much web maps solutions there are a quite simple way to load and show points, but when you are possibly implement it you will find out later that it is quite inefficient way whenever your points count became huge.

So, what i can do with it?

First you should consider using of some modern maps solutions like Mapbox. If you gonna consider Mapbox, you gonna save a lot of time, just because their algorithsm allow you to simply load and put in many of data.

Even if you don’t gonna use Mapbox for some reason, you should get familliar with what is popular in term of Maps Industry for nowaday called Vector Tiles. As term state by itself it is something that by itself a counterweight to Raster Tiles.

Small dig into maps

Typical Maps application loads chunks whenever your viewport hit them, even better, some of them can lazy load close ones to your viewport. In the case of Raster Tiles your browser (or app), loads ton of plain images for different level of zooms (discrete M to N). That was a quite good pattern for a while. It is easy to do caching in that way, but also that method has some limitations.

Vector Tiles are about a different way, your app are fed with a GIS information like geometries and how actually they should be dispalyed, even better your app can actually redefine how they are actually displaying. That pattern opens brave new world of possibilities. Maps now can be smooth scrolled with ease, not only by discrete levels of zoom, but also by any kinda sublevels of zoom, just because our data are rendered with client.

So, whenever you had a geometry (more general concept then just a point), you definitely should consider usage of a Vector Tiles. There are implementations for Mapbox (GL JS, etc), Leaflet (VectorGrid, etc) and more.

Okay, but what about points?

So strategy is quite simple, load just things what are inside boundaries of viewport projected to map.

On a backend you gonna implement some kind of tiles query processing, and then use some kind of GIS system, like PostGIS (also you want to smart cache that and do clustering 😉).

Useful stuff for PostGIS.

ST_MakePoint(longitude, latitude) @ ST_MakeEnvelope(?, ?, ?, ?, :srsid) -- point laying in boundaries

ST_AsMVT(tile) FROM (SELECT ST_AsMVTGeom(... -- generate mapbox vector tile output directly from postgis