Write a Python Vector Data Provider in PyQGIS
Every QGIS vector layer gets its features from a data provider: ogr for files, postgres for PostGIS, WFS for web services. When data lives somewhere none of them can reach — an in-house REST API, a proprietary sensor log, a Python object model in another application — the usual answer is to copy it into a memory layer or GeoPackage. Sometimes copying is the wrong answer: the data is large and only small parts are viewed at a time, or it changes constantly, or the layer must behave like a live source in projects. A provider written in Python makes the source appear as a normal layer, with QGIS asking for features as it needs them.
This recipe belongs to Extending QGIS with Custom Classes. It builds a read-only provider over a simple Python data source, implements the three cooperating classes a provider needs, honours feature requests so QGIS fetches only what it needs, registers the provider key, and compares the approach with a memory layer.
Prerequisites
- QGIS 3.34 LTR or newer, or the QGIS 4 series. Python providers have been possible since 3.12; the API shown follows QGIS's own Python provider tests.
- A data source accessible from Python. The example wraps a list of sensor records; in practice it might be a REST client or a database driver.
- Comfort with Python classes and with how layers read features, as in reading feature attributes and geometry.
The data source
The provider wraps something that can list records. Keeping that access in its own small class separates "how to talk to the source" from "how to be a QGIS provider".
class SensorSource:
"""Stand-in for a REST client or driver: records with id, x, y, attributes."""
def __init__(self, uri):
self.uri = uri # e.g. "https://sensors.example.org/api?network=air"
self._records = [
{"id": 1, "x": 13.40, "y": 52.52, "name": "Mitte", "pm25": 11.2},
{"id": 2, "x": 13.45, "y": 52.50, "name": "Friedrichshain", "pm25": 14.8},
{"id": 3, "x": 13.31, "y": 52.51, "name": "Charlottenburg", "pm25": 9.6},
]
def records(self, bbox=None, ids=None):
for r in self._records:
if ids is not None and r["id"] not in ids:
continue
if bbox and not (bbox[0] <= r["x"] <= bbox[2] and bbox[1] <= r["y"] <= bbox[3]):
continue
yield r
Breakdown: The records method accepts the two filters most sources can apply cheaply — a bounding box and a list of ids — so the provider can push them down instead of fetching everything. A real implementation would turn them into query parameters on an HTTP request or a WHERE clause. Ids must be stable integers, because QGIS uses them as feature ids.
The feature iterator
The iterator does the actual reading. QGIS calls fetchFeature repeatedly until it returns False; each call fills one feature. The iterator must honour the request — feature ids, a filter rectangle, an expression — or QGIS will draw and select the wrong features.
from qgis.core import (QgsAbstractFeatureIterator, QgsAbstractFeatureSource,
QgsFeatureIterator, QgsFeatureRequest, QgsFeature, QgsGeometry,
QgsPointXY, QgsCoordinateTransform)
class SensorIterator(QgsAbstractFeatureIterator):
def __init__(self, source, request):
super().__init__(request)
self._source = source
self._request = request if request is not None else QgsFeatureRequest()
self._transform = QgsCoordinateTransform()
dest = self._request.destinationCrs()
if dest.isValid() and dest != source.crs:
self._transform = QgsCoordinateTransform(source.crs, dest,
self._request.transformContext())
rect = self._request.filterRect()
bbox = None
if not rect.isNull():
if self._transform.isValid():
rect = self._transform.transformBoundingBox(rect, QgsCoordinateTransform.ReverseTransform)
bbox = (rect.xMinimum(), rect.yMinimum(), rect.xMaximum(), rect.yMaximum())
ids = None
if self._request.filterType() == QgsFeatureRequest.FilterFid:
ids = {self._request.filterFid()}
elif self._request.filterType() == QgsFeatureRequest.FilterFids:
ids = set(self._request.filterFids())
self._records = iter(list(source.client.records(bbox=bbox, ids=ids)))
def fetchFeature(self, f):
for r in self._records:
f.setFields(self._source.fields, True)
f.setId(r["id"])
f.setAttributes([r["id"], r["name"], r["pm25"]])
geom = QgsGeometry.fromPointXY(QgsPointXY(r["x"], r["y"]))
if self._transform.isValid():
geom.transform(self._transform)
f.setGeometry(geom)
f.setValid(True)
if self._request.filterType() == QgsFeatureRequest.FilterExpression:
self._request.expressionContext().setFeature(f)
if not self._request.filterExpression().evaluate(self._request.expressionContext()):
continue
return True
return False
def __iter__(self):
return self
def __next__(self):
feature = QgsFeature()
if not self.nextFeature(feature):
raise StopIteration
return feature
def rewind(self):
return False
def close(self):
self._records = iter(())
return True
Breakdown: The constructor translates the request into what the source understands: a bounding box (transformed back into the source CRS if QGIS asked for features in another CRS) and a set of ids. fetchFeature builds each feature with the provider's fields, a stable id, attributes in field order and a geometry, transformed to the destination CRS when requested — the map canvas often asks for features in the project CRS. Expression filters are evaluated per feature as a fallback, since most sources cannot evaluate QGIS expressions. __iter__ and __next__ make the iterator usable in Python for loops; rewind returning False tells QGIS to create a new iterator rather than restart this one.
The feature source and the provider
The feature source is a snapshot of everything the iterator needs, safe to hand to rendering threads. The provider describes the layer and creates sources and iterators.
from qgis.core import (QgsVectorDataProvider, QgsDataProvider, QgsFields, QgsField,
QgsWkbTypes, QgsCoordinateReferenceSystem, QgsRectangle)
from qgis.PyQt.QtCore import QVariant
class SensorFeatureSource(QgsAbstractFeatureSource):
def __init__(self, provider):
super().__init__()
self.client = provider._client
self.fields = provider.fields()
self.crs = provider.crs()
def getFeatures(self, request):
return QgsFeatureIterator(SensorIterator(self, request))
class SensorProvider(QgsVectorDataProvider):
KEY = "pysensors"
@classmethod
def providerKey(cls):
return cls.KEY
@classmethod
def description(cls):
return "Sensor network (Python provider)"
@classmethod
def createProvider(cls, uri, options, flags=QgsDataProvider.ReadFlags()):
return SensorProvider(uri, options, flags)
def __init__(self, uri="", options=QgsDataProvider.ProviderOptions(),
flags=QgsDataProvider.ReadFlags()):
super().__init__(uri)
self._uri = uri
self._client = SensorSource(uri)
self._fields = QgsFields()
self._fields.append(QgsField("id", QVariant.Int))
self._fields.append(QgsField("name", QVariant.String))
self._fields.append(QgsField("pm25", QVariant.Double))
self._crs = QgsCoordinateReferenceSystem("EPSG:4326")
def featureSource(self):
return SensorFeatureSource(self)
def getFeatures(self, request=QgsFeatureRequest()):
return QgsFeatureIterator(SensorIterator(SensorFeatureSource(self), request))
def dataSourceUri(self, expandAuthConfig=False):
return self._uri
def storageType(self):
return "Python sensor API"
def wkbType(self):
return QgsWkbTypes.Point
def fields(self):
return self._fields
def crs(self):
return self._crs
def featureCount(self):
return sum(1 for _ in self._client.records())
def extent(self):
rect = QgsRectangle()
for r in self._client.records():
rect.combineExtentWith(r["x"], r["y"])
return rect
def isValid(self):
return True
def name(self):
return self.KEY
def capabilities(self):
return QgsVectorDataProvider.SelectAtId
Breakdown: The feature source copies the provider's client, fields and CRS so iterators created in other threads never touch the provider itself. The provider's class methods give the registry its key, a description and a factory. extent and featureCount are computed from the source; for a large remote source, return cached or server-reported values instead, because QGIS calls them often. capabilities advertises only SelectAtId, so QGIS treats the layer as read-only and never offers editing. Fields, geometry type and CRS must be stable for the provider's lifetime.
Register the provider and load a layer
The provider registry needs metadata linking the key to the factory. After that, layers are created like any other.
from qgis.core import QgsProviderRegistry, QgsProviderMetadata, QgsVectorLayer, QgsProject
metadata = QgsProviderMetadata(SensorProvider.providerKey(), SensorProvider.description(),
SensorProvider.createProvider)
QgsProviderRegistry.instance().registerProvider(metadata)
layer = QgsVectorLayer("https://sensors.example.org/api?network=air", "air sensors",
SensorProvider.providerKey())
print(layer.isValid(), layer.featureCount(), layer.extent().toString(3))
QgsProject.instance().addMapLayer(layer)
Breakdown: Registration happens once per session — in a plugin's initGui, before any project using the provider opens. The layer source string is passed to the provider as its URI; design it to carry everything the provider needs, such as an endpoint and a network name, so projects can store and reload it. Once loaded, the layer works with styling, labels, the attribute table, expressions and Processing, because they all go through the iterator. Keep a reference to the metadata object for the session.
Test the provider like a layer
The best test of a provider is to use it the way QGIS will: iterate with filters, request another CRS, and check counts and extents.
from qgis.core import QgsRectangle
assert layer.featureCount() == 3
west = list(layer.getFeatures(QgsFeatureRequest().setFilterRect(QgsRectangle(13.2, 52.4, 13.35, 52.6))))
assert [f["name"] for f in west] == ["Charlottenburg"]
high = list(layer.getFeatures('"pm25" > 12'))
assert [f.id() for f in high] == [2]
req = QgsFeatureRequest().setDestinationCrs(QgsCoordinateReferenceSystem("EPSG:25833"),
QgsProject.instance().transformContext())
x = next(layer.getFeatures(req)).geometry().asPoint().x()
assert 380000 < x < 400000
print("provider behaves")
Breakdown: Each assertion exercises one part of the contract: the count, a spatial filter pushed to the source, an expression filter evaluated in the iterator, and a reprojection requested by the caller. These are exactly the requests the canvas, the identify tool, the attribute table and Processing will make. Adding them to a pytest suite, as in unit testing a QGIS plugin, protects the provider as it grows.
When a memory layer is simpler
A Python provider is the right tool when data is large and viewed piecemeal, when it changes and should be read live, or when the layer must reopen in projects pointing at its source. When the data is small and a snapshot is acceptable, a memory layer filled from the same client is far less code — see creating a memory layer. Python providers also run Python for every feature fetched, including during rendering, so they are slower than native providers; cache aggressively and push filters to the source.
QGIS version compatibility
Python vector data providers work on QGIS 3.34 LTR, 3.40 LTR and QGIS 4 using the pattern from QGIS's own provider test suite. On QGIS 4, enums are scoped — QgsFeatureRequest.FilterType.Fid, QgsVectorDataProvider.Capability.SelectAtId, Qgis.WkbType.Point — and QgsField takes QMetaType.Type values. Check the exact enum names on your target version with help().
Troubleshooting
- The layer is invalid. The provider key was not registered before the layer was created, or
isValidreturned False. - Features appear in the wrong place. The iterator ignored
destinationCrsand returned source coordinates. - Selections or identify return the wrong feature. Feature ids are not stable or the fid filters are ignored.
- QGIS crashes during rendering. The iterator touches the provider object from a worker thread; use the feature source snapshot.
Conclusion
Wrap the data access in a small client, write an iterator that honours fid, rectangle, expression and CRS requests, hand iterators a thread-safe feature source, describe fields, geometry type, CRS, extent and capabilities in the provider, register the key from a plugin, and test with the same requests QGIS will make — or use a memory layer when a snapshot is enough.
Frequently Asked Questions
Can a Python provider support editing?
Yes, by implementing addFeatures, changeAttributeValues and friends and advertising the capabilities. Start read-only and add editing only if needed.
Will the layer work in QGIS Server? Only where the plugin registering the provider is loaded on the server.
How do I add a source-select dialog?
Implement a QgsSourceSelectProvider so users can add layers from the Data Source Manager.
Is a Python provider faster than a memory layer? No, usually slower per feature. Its advantage is fetching only what is needed, live.