Layer Data Sources & Formats in PyQGIS

Every layer in QGIS is two things: a provider that knows how to read a kind of data, and a source string that tells that provider where to find it. Most PyQGIS scripts treat the second part as an opaque path passed to QgsVectorLayer and never think about it again — until a CSV arrives with decimal commas, a GeoPackage turns out to hold forty tables, a file server is renamed, or somebody needs a join across a spreadsheet and a database table without copying either.

This guide sits inside Spatial Data Processing & Automation and covers the layer between your script and your files. It explains how providers and source strings work, how to read and build them safely, how to handle containers that hold many layers, and how to get data in from the formats that are not really GIS formats at all: spreadsheets, SQL views and photographs.

Providers and source strings

A layer object does very little reading of its own. When you write QgsVectorLayer("/data/roads.gpkg|layername=roads", "roads", "ogr"), the third argument is a provider key, and the provider registry uses it to create a data provider — the object that actually opens files, runs queries, and iterates features. The layer then delegates to it. Styling, labelling, expressions, Processing and layouts all talk to the layer, which is why they work identically whether the features came from a shapefile, a PostGIS table or a WFS.

The source string is the provider's private language. The ogr provider expects a path with optional |layername= and |subset= suffixes. The delimitedtext provider expects a file:// URL with query parameters. Database providers expect key='value' pairs; web providers expect URL-encoded parameter lists. None of these formats is documented as a stable public contract, and all of them have escaping rules that break naive string handling.

Two rules follow. First, always check isValid() after creating a layer, because a provider that cannot open its source still produces a layer object — it just has no data. Second, never assemble or pick apart a source string with split and f-strings when the registry can do it for you. Decoding and building data source URIs covers decodeUri, encodeUri and QgsDataSourceUri, which between them handle every provider family.

from qgis.core import QgsProject, QgsProviderRegistry

registry = QgsProviderRegistry.instance()
print(sorted(registry.providerList()))

for layer in QgsProject.instance().mapLayers().values():
    print(f"{layer.name():<28} {layer.providerType():<14} valid={layer.isValid()}")
    print("    ", registry.decodeUri(layer.providerType(), layer.source()))

Breakdown: providerList() shows which providers your installation actually has — a minimal container image may lack wms or mssql, which explains an invalid layer faster than any error message. Printing the provider type beside each layer is the fastest way to understand an unfamiliar project; decoding the source shows you the parts a script is likely to need, such as the file path behind each layer.

When a layer refuses to open, the provider's error and the underlying GDAL message are where the reason lives; debugging invalid layer loading walks through reading them.

Text files that describe geometry

The most common input that is not a GIS format is a spreadsheet export. The delimitedtext provider reads CSV and other delimited files in place, turning two coordinate columns or one WKT column into geometry. It is fast, it keeps the file as the single source of truth, and it has two properties that surprise people: it is read-only, and it never rejects a row. A row with a blank coordinate or an unparseable number becomes a feature with null geometry — present in the attribute table, invisible on the map, and uncounted unless you look.

The defence is to be explicit about everything the provider would otherwise guess — delimiter, decimal separator, encoding, coordinate fields, geometry type and CRS — and to count null geometries immediately after loading. Loading a CSV as a point layer builds the URI from a dictionary, handles European number formats and WKT columns, and copies the result into GeoPackage once it looks right.

Photographs are a text source of a different kind: the coordinates live in each file's EXIF header rather than in a column. native:importphotos reads a whole folder of them into a point layer with direction and timestamp fields, and importing geotagged photos adds the reconciliation step that catches photos without a fix, plus rotated markers and photo map tips.

from pathlib import Path
from urllib.parse import urlencode
from qgis.core import QgsVectorLayer, QgsFeatureRequest

params = {"type": "csv", "delimiter": ";", "decimalPoint": ",",
          "detectTypes": "yes", "xField": "x", "yField": "y",
          "crs": "EPSG:25832"}
uri = f"{Path('/data/inbox/messpunkte.csv').as_uri()}?{urlencode(params)}"
points = QgsVectorLayer(uri, "messpunkte", "delimitedtext")

empty = sum(1 for _ in points.getFeatures(
    QgsFeatureRequest().setFilterExpression("$geometry IS NULL")))
print(points.featureCount(), "rows,", empty, "without geometry")

Breakdown: Every parameter that affects the result is stated, so the same script gives the same layer on any machine and any locale. The null-geometry count is the number that tells you whether the parsing assumptions were right; if it is not zero, the reason is almost always a separator or a blank cell, and the rows can be listed by identifier.

Containers: one file, many layers

GeoPackage, File Geodatabase, KML, spreadsheets with several sheets and multi-variable NetCDF files are containers. Opening the file is not the same as opening a layer; the source string has to name which table, sheet or subdataset you mean, and hard-coding that name ties the script to whatever the supplier called it last time.

QgsProviderRegistry.querySublayers solves that. It asks every capable provider what the file contains and returns a details object per sublayer, carrying the name, type, geometry and a ready-to-use URI. Filtering those details and calling toLayer on the survivors loads exactly what you need, with the right provider, without ever opening the tables you skipped. Listing and loading GeoPackage sublayers covers fast scans versus counted queries, mixed vector and raster content, and the default styles a GeoPackage can carry.

from qgis.core import (
    Qgis, QgsProject, QgsProviderRegistry, QgsProviderSublayerDetails, QgsWkbTypes,
)

path = "/data/deliveries/april/base.gpkg"
options = QgsProviderSublayerDetails.LayerOptions(
    QgsProject.instance().transformContext())

roads = [d for d in QgsProviderRegistry.instance().querySublayers(path)
         if d.type() == Qgis.LayerType.Vector
         and "road" in d.name().lower()
         and QgsWkbTypes.geometryType(d.wkbType()) == Qgis.GeometryType.Line]
if len(roads) != 1:
    raise RuntimeError(f"expected one road table, found {[d.name() for d in roads]}")
road_layer = roads[0].toLayer(options)

Breakdown: Filtering by geometry type as well as by name stops a road_names attribute table or a road_buffers polygon layer being picked up by accident. Insisting on exactly one match turns an ambiguous delivery into a clear error instead of a silently wrong choice — the right behaviour for a scheduled batch job that nobody is watching.

Writing into containers is the other half. Adding a table to an existing GeoPackage without overwriting the file needs the right writer options, which writing a vector layer to GeoPackage covers in detail.

Views: SQL across layers

Sometimes the data you need does not exist as a table anywhere: a count of incidents per ward, a spreadsheet of readings joined to station locations, a filtered union of two layers. The virtual provider builds that as a live SQL view over any loaded layers, using SQLite with SpatiaLite functions, and exposes it as an ordinary read-only layer.

Virtual layers are powerful and easy to make slow. The query runs every time the layer is read — on every pan, identify and attribute table refresh — so declaring the unique id and geometry up front, and using the _search_frame_ column to push spatial joins onto an index, is the difference between an instant layer and a frozen canvas. Querying layers with virtual layer SQL builds definitions in Python, reads them back from saved projects, and shows when Execute SQL or a query in PostGIS is the better choice.

from qgis.core import QgsVectorLayer, QgsVirtualLayerDefinition

definition = QgsVirtualLayerDefinition()
definition.setQuery("""
    SELECT s.fid AS station_id, s.name, max(r.pm25) AS peak_pm25, s.geometry
    FROM stations AS s JOIN readings AS r ON r.station = s.code
    GROUP BY s.fid
""")
definition.setUid("station_id")
definition.setGeometryField("geometry")
peaks = QgsVectorLayer(definition.toString(), "peak PM2.5", "virtual")
print(peaks.isValid(), peaks.featureCount())

Breakdown: stations and readings are the names of layers in the project — one a point layer from a database, the other a geometryless CSV — and the provider resolves them at open time. The integer uid keeps feature ids stable, so a selection on the map survives a refresh.

When the data moves

Projects store directions to data, not data. That is what makes them small and shareable, and it is also why they break when a file server is reorganised, a drive letter changes or a database moves to a new host. QGIS Desktop offers a dialog to fix unavailable layers one project at a time; at the scale of a migration, you need the same repair as a script.

The repair has a clear shape. Read each project with layer resolution turned off so nothing tries to open the missing data; decode each layer's source; apply an ordered table of old-to-new prefixes; confirm the new path exists; and apply it with setDataSource, which keeps styles, joins, relations and layout references attached. Fixing broken layer paths in a project builds that script, adds a dry-run audit across a whole folder of projects, and saves the result with relative path storage.

from qgis.core import Qgis, QgsProject

project = QgsProject.instance()
project.read("/srv/projects/flood.qgz", Qgis.ProjectReadFlag.DontResolveLayers)
unresolved = {
    layer.name(): registry.decodeUri(layer.providerType(), layer.source()).get("path")
    for layer in project.mapLayers().values()
}
print(unresolved)

Breakdown: Reading without resolving is fast even when the share is gone, because no provider opens anything; every layer is a placeholder that still carries its full definition. The dictionary it produces is the input to the remapping step, and printing it before changing anything is the cheapest possible audit of what a project depends on.

Provider capabilities decide what a script may do

Loading a layer is only the start; what you can do with it next depends on the provider behind it. A delimitedtext layer cannot be edited. A WFS layer can be edited only if the server supports transactions. A virtual layer cannot add fields. A GeoPackage layer can do nearly everything, and a shapefile can do most things but will silently truncate a new field name to ten characters. Scripts that assume every layer behaves like a GeoPackage fail half-way through a batch, after some changes have been written.

Every data provider reports what it supports as a set of capability flags, and checking them before starting work turns those mid-batch failures into a clear message up front.

from qgis.core import QgsVectorDataProvider

def require(layer, *caps):
    provider = layer.dataProvider()
    have = provider.capabilities()
    missing = [name for name, cap in caps if not have & cap]
    if missing:
        raise RuntimeError(
            f"{layer.name()} ({layer.providerType()}) cannot: {', '.join(missing)}. "
            f"It supports: {provider.capabilitiesString()}"
        )

require(points,
        ("add features", QgsVectorDataProvider.AddFeatures),
        ("change attribute values", QgsVectorDataProvider.ChangeAttributeValues),
        ("add fields", QgsVectorDataProvider.AddAttributes))

Breakdown: capabilities() returns a bit mask, and testing each required flag against it tells you before any edit whether the layer can accept it. For the CSV layer loaded earlier in this guide the check fails immediately on AddFeatures, which is the prompt to copy it into GeoPackage. capabilitiesString() gives a human-readable list of everything the provider does support, which is worth logging once per layer in unattended jobs. The same idea applies to spatial filtering: providers that report SelectAtId and a native spatial index answer QgsFeatureRequest filters quickly, while a provider without them falls back to reading everything, which is why speeding up feature iteration starts by asking where the data lives.

Choosing the right route in

Most format decisions come down to three questions: whether the data should stay where it is, whether it will be edited, and whether a script will run against it repeatedly.

Keep it in place, read it often: load it directly — a delimitedtext layer over a CSV, a sublayer from a GeoPackage, a WFS. The source stays authoritative.
Edit it, or analyse it seriously: copy it into a GeoPackage first. You gain real field types, a persistent spatial index and edit support, and you stop depending on re-detection of a text file.
Combine several sources without copying: a virtual layer, accepting that it recomputes on read.
Combine and keep the answer: Execute SQL or a dedicated Processing algorithm, writing a real output.
Data that lives on a server: let the server do the work where it can — PostGIS queries, or filters pushed into WFS requests.

The order of those questions matters. Editing trumps everything else, because a read-only provider cannot be made writable after the fact and discovering that half-way through a field campaign is expensive. Combining sources comes next, because it determines whether a single provider is even involved. Only then does freshness decide between a view and a snapshot. A script that encodes the same order — checking capabilities first, then counting sources, then choosing a view or an output — makes the same decision every time a new dataset arrives, which is ultimately what separates an automated pipeline from a collection of one-off imports.

Key takeaways

A layer is a provider key plus a source string; always check isValid() after creating one.
Decode and encode source strings with the provider registry and QgsDataSourceUri, never with string splitting.
delimitedtext never rejects a row — count null geometries after every CSV load.
Query containers with querySublayers and filter by type, geometry and name instead of hard-coding table names.
Virtual layers are live SQL views; declare the uid and geometry and use _search_frame_ for spatial joins.
Repair moved projects by reading them unresolved, remapping prefixes, applying setDataSource, and saving with relative paths.

Frequently Asked Questions

Which format should intermediate results use? GeoPackage for anything you keep, memory layers for anything you discard at the end of the script. Shapefiles truncate field names to ten characters, cap file size at 2 GB, cannot store null dates properly and split into several files that travel badly by email; CSV loses types and geometry precision. FlatGeobuf is a good choice for large read-only outputs that will be streamed over HTTP, and GeoParquet for handing data to analytics tools outside QGIS where your GDAL build supports it.

Why is my layer valid but empty? The source opened but a subset string, a wrong layername, or a query returning no rows filtered everything out. Print layer.subsetString() and decode the source to see which.

Can a layer switch provider without being removed? Yes. setDataSource takes a provider key, so a shapefile layer can be repointed at a PostGIS table and keep its style, provided the geometry type and fields are compatible.

Do these techniques work in standalone scripts and QGIS Server? Yes. Providers, the registry and project reading are all part of qgis.core and need no GUI.

How do I know which parameters a provider's URI accepts? Build a layer the way you want it in the Data Source Manager, then print layer.source() and decode it. That is the most reliable documentation for your installed version.