Batch Process All Layers in a Project with PyQGIS

Many batch jobs start not from a folder of files but from a project someone has already assembled: forty layers in carefully named groups, each loaded from a different source with its own subset filter. "Reproject everything in this project", "export every layer in the Deliverables group to GeoPackage", "fix geometries in all polygon layers", "add a checksum field to every table" — the project is the unit of work, and the script should respect the structure people put into it.

This recipe belongs to Batch Processing with PyQGIS. It walks a project's layers by type, group and name, skips layers that do not apply, runs a Processing algorithm per layer, writes outputs with sensible names, optionally swaps results back into the project, and reports per layer.

Selecting layers from a projectA project's layer tree contains groups such as Base, Analysis and Deliverables, each holding layers of different types. A selection step walks the tree and keeps layers matching rules: inside a given group, of vector polygon type, with names matching a pattern, and valid. The selected layers are processed one by one and each result is recorded in a report.Walk the tree, apply rules, processlayer treeBase/Analysis/Deliverables/groups + layersrulesin group?type = polygon?name pattern?valid?per layerprocesswrite outputreport line

Prerequisites

  • QGIS 3.34 LTR or newer, or the QGIS 4 series.
  • A project, either open in QGIS or loaded in a standalone script as in saving and loading a QGIS project.
  • A clear rule for which layers to process; "all of them" is rarely right.

Iterate layers in tree order

QgsProject.mapLayers() returns layers in no particular order. Walking the layer tree instead follows the order and grouping users see in the Layers panel, which makes reports and outputs easier to follow.

from qgis.core import QgsProject, QgsLayerTreeLayer, QgsLayerTreeGroup

project = QgsProject.instance()

def layers_in_tree(node=None, path=()):
    node = node or project.layerTreeRoot()
    for child in node.children():
        if isinstance(child, QgsLayerTreeGroup):
            yield from layers_in_tree(child, path + (child.name(),))
        elif isinstance(child, QgsLayerTreeLayer) and child.layer() is not None:
            yield "/".join(path), child.layer()

for group, layer in layers_in_tree():
    print(f"{group or '(root)':<20} {layer.name():<28} {type(layer).__name__}")

Breakdown: Recursing through groups yields each layer with its group path, such as Deliverables/Roads, which is useful both as a selection criterion and for building output folder names. Tree layers whose layer() is None are broken references — layers removed from the project but left in the tree — and are skipped. A layer that appears twice in the tree is yielded twice; collect ids in a set if each should be processed once.

Select the layers that apply

Combining a few rules — group, type, geometry, name pattern, validity — selects exactly the layers a job should touch.

Selection rulesA layer is selected when it is valid, is a vector layer with polygon geometry, sits in the Deliverables group or below, and its name does not start with an underscore, a convention for scratch layers. Each skipped layer is recorded with the reason, so the report explains why something was not processed.Keep, or skip with a reasonkeep ifvalid vector, polygonunder Deliverables/name not _scratchelse skip with reasoninvalid sourcewrong type or groupreported, not silent

import re
from qgis.core import QgsVectorLayer, QgsWkbTypes

def select(group, layer, group_prefix="Deliverables", pattern=r"^[^_]"):
    if not layer.isValid():
        return False, "invalid source"
    if not isinstance(layer, QgsVectorLayer):
        return False, "not a vector layer"
    if layer.geometryType() != QgsWkbTypes.PolygonGeometry:
        return False, "not polygons"
    if not group.startswith(group_prefix):
        return False, f"outside {group_prefix}"
    if not re.match(pattern, layer.name()):
        return False, "scratch layer"
    return True, ""

selected, skipped = [], []
for group, layer in layers_in_tree():
    ok, why = select(group, layer)
    (selected if ok else skipped).append((group, layer, why))
print(len(selected), "selected;", len(skipped), "skipped")

Breakdown: Each rule returns a reason when it rejects a layer, so the final report can say why every skipped layer was skipped — far more useful than a silent filter when someone asks why their layer was not exported. A naming convention such as a leading underscore for scratch layers lets users opt layers out without editing the script. Rules are ordinary Python and easy to change per job.

Run an algorithm per layer

Each selected layer becomes an input to the same Processing algorithm. The layer object can be passed directly, which preserves subset filters and joined fields — something a file path would lose.

import processing
from pathlib import Path

out_dir = Path("/data/exports") / project.baseName()
out_dir.mkdir(parents=True, exist_ok=True)
report = []

for group, layer, _ in selected:
    safe = re.sub(r"[^A-Za-z0-9_-]+", "_", f"{group}_{layer.name()}").strip("_")
    try:
        fixed = processing.run("native:fixgeometries", {
            "INPUT": layer, "OUTPUT": "memory:"})["OUTPUT"]
        out = processing.run("native:savefeatures", {
            "INPUT": fixed, "OUTPUT": str(out_dir / "deliverables.gpkg"),
            "LAYER_NAME": safe, "ACTION_ON_EXISTING_FILE": 1})["OUTPUT"]
        report.append((layer.name(), "ok", fixed.featureCount(), out))
    except Exception as err:
        report.append((layer.name(), f"FAILED: {err}", 0, ""))

Breakdown: Passing the layer object means Processing reads exactly what the user sees — filtered, with joins — rather than the raw source file. Output table names combine group and layer names, cleaned of characters GeoPackage dislikes, so Deliverables/Roads (2026) becomes Deliverables_Roads_2026. ACTION_ON_EXISTING_FILE: 1 adds each table to the same GeoPackage; make sure the file starts fresh or uses mode 0 on the first write if reruns should replace it. Catching exceptions per layer keeps one bad layer from stopping the rest.

Swap results back into the project

Some jobs should change the project itself: replace each layer with its repaired or reprojected version while keeping styles, names and positions in the tree.

for group, layer, _ in selected:
    name = layer.name()
    safe = re.sub(r"[^A-Za-z0-9_-]+", "_", f"{group}_{name}").strip("_")
    new_source = f"{out_dir / 'deliverables.gpkg'}|layername={safe}"
    layer.setDataSource(new_source, name, "ogr")
    if not layer.isValid():
        print("could not repoint", name)
project.write(str(out_dir / f"{project.baseName()}_fixed.qgz"))

Breakdown: setDataSource repoints an existing layer to a new source while keeping its id, style, labels, forms and position in the tree — the same technique used in fixing broken layer paths. Writing the result as a new project file leaves the original untouched, which is the right default; overwrite only after checking the copy. Joins and relations referencing the layer by id keep working, because the id does not change.

Run safely on a project people are using

Running a batch in the Python console against an open project has risks a standalone script does not: layers in edit mode, unsaved changes, a user still working. A few guards make it safe.

from qgis.utils import iface

editing = [l.name() for _, l, _ in selected if l.isEditable() or l.isModified()]
if editing:
    raise RuntimeError(f"save or close edits first: {editing}")
if project.isDirty():
    iface.messageBar().pushWarning("Batch", "Project has unsaved changes — save a copy before swapping sources")

backup = out_dir / f"{project.baseName()}_before_batch.qgz"
project.write(str(backup))
print("backup written to", backup)

Breakdown: Refusing to run while any selected layer is in edit mode avoids processing a mix of saved and unsaved features — Processing reads the edit buffer for layer objects, which may or may not be what the user intends. Writing a backup copy of the project before swapping data sources makes the change reversible in one step. For long batches inside QGIS, move the work into a background task so the interface stays responsive, applying source swaps on the main thread when it finishes.

Report per layer

A batch over a project should end with a short report: every layer, what happened, how many features, where the output went.

One line per layerThe report lists every processed layer with status ok or failed with the message, the number of features written and the output location, followed by every skipped layer with the reason. Written as CSV next to the outputs, it answers questions about the run without rerunning it.Processed and skipped, both explainedprocessedlayer · statusfeatures · outputok or error textskippedlayer · reasoninvalid, type, group

import csv

with open(out_dir / "batch_report.csv", "w", newline="", encoding="utf-8") as fh:
    w = csv.writer(fh)
    w.writerow(["layer", "status", "features", "output"])
    w.writerows(report)
    for group, layer, why in skipped:
        w.writerow([layer.name(), f"skipped: {why}", "", ""])
failed = [r for r in report if r[1] != "ok"]
print(f"{len(report) - len(failed)} ok, {len(failed)} failed, {len(skipped)} skipped")

Breakdown: Including skipped layers with their reasons makes the report a complete account of the project, not just of what succeeded. Failures carry the exception message, which is usually enough to fix the cause and rerun only those layers. For runs that take long enough to be interrupted, combine this with checkpointing a batch run.

Run against many projects

The same function applied to a folder of projects processes an organisation's whole project collection — every project's deliverables exported nightly, for example.

from qgis.core import QgsProject

for qgz in sorted(Path("/data/projects").glob("*.qgz")):
    p = QgsProject.instance()
    p.clear()
    if not p.read(str(qgz)):
        print("cannot read", qgz.name)
        continue
    print("processing", qgz.name)
    # call the selection and processing steps above here

Breakdown: Clearing and reading each project in turn reuses the same project instance; everything above then runs against the loaded project. Projects with broken layer paths load with invalid layers, which the validity rule skips and reports. In standalone scripts, read projects with flags that skip loading layouts or data you do not need for speed, as in opening a project with read flags.

QGIS version compatibility

Layer tree traversal, setDataSource and Processing calls work on QGIS 3.34 LTR, 3.40 LTR and QGIS 4. setDataSource with provider options has existed since 3.20. On QGIS 4, QgsWkbTypes.PolygonGeometry is Qgis.GeometryType.Polygon.

Troubleshooting

  • Layers are processed in a random order. mapLayers() was used; walk the layer tree instead.
  • Subset filters were ignored. File paths were passed instead of layer objects.
  • Output tables overwrite each other. Names collide after cleaning; include the group path or a counter.
  • The project lost styles after swapping. A new layer was added instead of repointing; use setDataSource.

Conclusion

Walk the layer tree to follow the project's order and groups, select layers with explicit rules that record reasons for skipping, pass layer objects to Processing so filters and joins are respected, write outputs with names derived from group and layer, repoint layers with setDataSource when the project should change, and finish with a per-layer report.

Frequently Asked Questions

Can I process only visible layers? Yes: check QgsLayerTreeLayer.isVisible() in the tree walk.

How do I include raster layers? Extend the rules to accept QgsRasterLayer and use raster algorithms for them.

Does this work in the Python console? Yes, against the open project; save a copy before swapping sources.

Can a model do this? Models iterate features, not project layers; a script is the right tool.