Use Memory Layers Between Algorithms in PyQGIS

A chain of four algorithms that writes four shapefiles produces three files nobody wants, in a folder somebody has to clean up, at the cost of three full round trips through the disk. Processing has a better answer built in: an output that lives in memory, is passed straight to the next step, and disappears when the script ends.

This recipe belongs to Chaining Processing Algorithms. It covers temporary outputs, passing results between steps, when memory is the wrong choice, keeping intermediate layers alive long enough to use them, and constructing a memory layer directly when no algorithm produced it.

Four steps, one file worth keepingWriting every step to disk produces three intermediate files that must be named, stored and cleaned up, and each step pays the cost of writing and re-reading. Keeping intermediates in memory passes each result directly to the next algorithm, so only the final output reaches the disk and nothing needs cleaning up.Only the last result is worth a file nameto diskbuffer.shpclipped.shpdissolved.shpresult.gpkgthree files to name, store and delete — and six disk traversalsin memorybufferedclippeddissolvedresult.gpkgnothing to clean up, and the intermediates never touch the diskUntil an intermediate no longer fits in memory — then the first row is right

Prerequisites

  • QGIS 3.34 LTR (bundled Python 3.12) or newer.
  • The processing module available — in the console it already is; in a standalone script it needs initialising.
  • A chain worth chaining; the single-algorithm case is in Run a Processing Algorithm from a Script.

Ask for a temporary output

import processing

buffered = processing.run("native:buffer", {
    "INPUT": "/data/city.gpkg|layername=roads",
    "DISTANCE": 25,
    "SEGMENTS": 8,
    "DISSOLVE": False,
    "OUTPUT": "TEMPORARY_OUTPUT",
})["OUTPUT"]

print(buffered)          # a layer id or a memory URI, depending on the algorithm

Breakdown: TEMPORARY_OUTPUT tells the sink to create a temporary destination rather than a file, and for most vector algorithms that means a memory layer. The returned dictionary is keyed by the algorithm's output names, so ["OUTPUT"] extracts the one you want; what it contains is either a layer object or a string identifying one, and the useful property is that it can be passed straight back into the next algorithm either way. Omitting OUTPUT entirely has the same effect for many algorithms, but naming it explicitly documents the intent and avoids surprises with algorithms whose default is a file.

Chain the steps

clipped = processing.run("native:clip", {
    "INPUT": buffered,
    "OVERLAY": "/data/city.gpkg|layername=boundary",
    "OUTPUT": "TEMPORARY_OUTPUT",
})["OUTPUT"]

dissolved = processing.run("native:dissolve", {
    "INPUT": clipped,
    "FIELD": ["road_class"],
    "OUTPUT": "TEMPORARY_OUTPUT",
})["OUTPUT"]

processing.run("native:reprojectlayer", {
    "INPUT": dissolved,
    "TARGET_CRS": "EPSG:4326",
    "OUTPUT": "/data/outputs/road_buffers.gpkg",
})

Breakdown: Each step takes the previous result as its input, and only the last one names a file. The chain reads top to bottom as the sequence of operations, which is most of its value — a script that writes and re-reads three shapefiles hides the same logic under file handling. Note the field list in the dissolve: several algorithms take lists where the interface shows a single choice, and passing a bare string there is a common cause of an unexpected result rather than an error.

Keep the layers alive

The one real trap: a temporary layer can be garbage-collected while your script still intends to use it.

from qgis.core import QgsProcessingContext, QgsProcessingFeedback

context = QgsProcessingContext()
feedback = QgsProcessingFeedback()

buffered = processing.run("native:buffer", {...},
                          context=context, feedback=feedback, is_child_algorithm=True)["OUTPUT"]

clipped = processing.run("native:clip", {"INPUT": buffered, ...},
                         context=context, feedback=feedback, is_child_algorithm=True)["OUTPUT"]

# the context owns the temporary layers until it goes out of scope
final = context.getMapLayer(clipped)

Breakdown: A shared QgsProcessingContext owns the temporary layers produced during the chain, which is what keeps them alive between calls — without one, a temporary layer can be released as soon as the local reference to it goes away, and the next algorithm reports that its input is invalid. is_child_algorithm=True says these are intermediate steps, so their outputs are not added to the project. context.getMapLayer() converts a returned layer id back into a real layer object at the end. In a plugin or a Processing algorithm you already have a context and should pass it down rather than creating one, which is the pattern in Report Progress and Cancellation in a Processing Algorithm.

What the context is forWithout a shared context, each temporary layer is owned only by the local variable holding it, and can be released before the next algorithm reads it. With a shared context passed to every call, the context owns each temporary layer for the whole chain, so every step's input remains valid until the context itself goes out of scope.Somebody has to own an intermediate layerone QgsProcessingContext, passed to every callholds every temporary layer the chain producesbufferedvalid through step 2clippedvalid through step 3dissolvedvalid to the end

When memory is the wrong choice

Temporary outputs are the default for a reason, and there are three situations where they are not what you want.

The intermediate does not fit. A memory layer holds every feature and geometry in RAM. A dissolve producing one enormous multipolygon, or a buffer over ten million points, will exhaust memory and take the process with it. Write those to a GeoPackage and let the operating system page it.

You need to inspect the intermediate. When a chain produces the wrong answer, the fastest diagnosis is looking at step two. Temporarily naming each output as a file — and loading them — turns a mystery into an obvious step.

The step is expensive and repeated. A costly intermediate that several later runs need should be computed once and written, not rebuilt on every run. That is a cache, and it belongs on disk with a note about when it was made.

A pragmatic compromise for development is a flag: run with temporary outputs normally, and with named files when a DEBUG switch is set. It costs a few lines and makes the chain inspectable exactly when you need it.

Temporary or file, decided per stepA step whose result fits comfortably in memory, is not being inspected, and is not reused elsewhere should be a temporary output. A result too large for memory, one you are currently debugging, or one that several later runs need should be written to a file instead, with the last case treated explicitly as a cache.Three questions, asked per stepdoes it fit in memory?no — write a filelet the disk page itare you inspecting it?yes — name itand load it to lookTEMPORARY OUTPUTnothing to clean up

Build a memory layer by hand

Where no algorithm produced the data — features computed in Python, rows read from an API — construct the layer directly.

from qgis.core import QgsVectorLayer, QgsFeature, QgsGeometry, QgsPointXY, QgsField
from qgis.PyQt.QtCore import QVariant

layer = QgsVectorLayer("Point?crs=EPSG:27700&field=ref:string&field=value:double",
                       "computed", "memory")
provider = layer.dataProvider()

features = []
for ref, x, y, value in readings:
    feature = QgsFeature(layer.fields())
    feature.setAttributes([ref, value])
    feature.setGeometry(QgsGeometry.fromPointXY(QgsPointXY(x, y)))
    features.append(feature)

provider.addFeatures(features)
layer.updateExtents()

Breakdown: The memory provider's URI declares geometry type, coordinate system and fields in one string, so no separate schema step is needed — and the CRS must be declared here, because a memory layer with no CRS silently participates in transformations as if it were in the project's. Constructing each feature from layer.fields() gives it the right attribute count; a bare QgsFeature() accepts attributes and drops them. Adding features in one addFeatures() call rather than one at a time is substantially faster on large sets. updateExtents() is needed before anything asks the layer for its extent, including most algorithms. This layer can now be passed to processing.run() exactly like any other input.

QGIS version compatibility

QGIS versionPythonNotes
3.22 LTR3.9TEMPORARY_OUTPUT, contexts and the memory provider all as described.
3.28 LTR3.9Identical.
3.34 LTR3.12Baseline for this page.
3.40 / 3.443.12Identical; some algorithms now default to temporary outputs where they previously required a destination.

Older examples use "memory:" as the output value, which still works and is equivalent to TEMPORARY_OUTPUT for vector sinks. The named constant is clearer and applies to raster outputs too, where it produces a temporary file rather than a memory layer.

Troubleshooting

  • "Input layer is not valid" on the second step. The intermediate was released. Share a QgsProcessingContext across the calls.
  • Memory use grows until the process dies. An intermediate is too large for RAM. Write that step to a GeoPackage.
  • The result is empty and no error appears. An earlier step produced nothing — often a clip with no overlap. Inspect the intermediates.
  • A field list parameter is ignored. It was passed as a string instead of a list.
  • Layers pile up in the project. is_child_algorithm was not set, so intermediate outputs were added. Set it for every step but the last.
  • A memory layer's features have no attributes. They were built with QgsFeature() rather than QgsFeature(layer.fields()).

Conclusion

Pass TEMPORARY_OUTPUT for every step whose result you do not want to keep, feed each result straight into the next algorithm, and name a real file only for the final output. Share one QgsProcessingContext across the chain so intermediates stay alive, mark intermediate calls as child algorithms so they do not clutter the project, and fall back to files when a step is too large for memory or when you need to look at it.

Frequently Asked Questions

Is a temporary output always a memory layer? For vector sinks, usually yes. Raster outputs go to a temporary file, since raster data in memory is rarely practical.

Do temporary layers appear in the project? Only if you add them, or if you run without is_child_algorithm. In the toolbox they appear as scratch layers marked as temporary.

Can I save a memory layer later? Yes — run it through an algorithm with a file output, or write it with QgsVectorFileWriter. Nothing about a memory layer prevents it being saved.

How much memory does a memory layer use? Roughly the size of the data plus overhead per feature. A few hundred thousand simple features is comfortable; complex geometry costs far more.

Are memory layers thread-safe? Treat them as not. Build them on one thread and hand the finished layer over, rather than writing from a background task.

Why does my chain work in the console and fail in a standalone script? Processing was not initialised, or nothing owns the temporary layers. Both are addressed in Running Python Scripts Outside QGIS Desktop.