Handle Errors and Logging in Unattended PyQGIS Scripts

The worst outcome for a scheduled job is not a crash. A crash gets noticed. The worst outcome is a run that completes in four seconds instead of four minutes, writes an empty layer, exits zero, and keeps doing that every night for six weeks until someone opens the map and asks where the data went.

This recipe belongs to Headless QGIS and Server Automation. It covers capturing the messages QGIS writes to its own log, feeding Processing's progress into Python logging, asserting that a result is plausible before declaring success, and using exit codes a scheduler can act on.

Four sources of truth, one logPython's own logging, the QGIS message log, Processing feedback objects and provider error messages each carry information about a run. Left alone they go to four different places, three of which are invisible on a server. A single handler collects all four into one structured log file, and the outcome sets the process exit code.Three of these are invisible unless you connect themlogging.getLogger()visible by defaultQgsMessageLoggoes to a GUI panel that is not thereQgsProcessingFeedbackdiscarded unless you pass oneprovider.error()only if you read itone handlertimestamp · levelcounts · durationnightly.logone line per unit of workexit code0 good · 1 error · 2 implausible

Prerequisites

  • A headless PyQGIS script — see Headless QGIS and Server Automation for the initialisation pattern.
  • A writable log location, and a scheduler that watches exit codes.
  • Familiarity with Python's logging module; nothing beyond basicConfig and a logger per module is needed.

Set up logging before anything else

import logging
import sys

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)-7s %(name)s %(message)s",
    datefmt="%Y-%m-%dT%H:%M:%S",
    stream=sys.stdout,
)
log = logging.getLogger("nightly_export")

Breakdown: Writing to standard output rather than opening a file lets the wrapper script or the container runtime decide where the log goes — the same script then works under cron, systemd and Docker without change. A fixed-width level column makes the log greppable; ISO timestamps make it sortable and unambiguous across time zones. Configure this before importing anything from QGIS so that early provider warnings are already captured.

Capture what QGIS itself is saying

QgsMessageLog is where providers, algorithms and plugins report problems. On the desktop it fills a panel; on a server it goes nowhere unless you connect to it.

from qgis.core import QgsApplication, Qgis

LEVELS = {
    Qgis.Info: logging.INFO,
    Qgis.Warning: logging.WARNING,
    Qgis.Critical: logging.ERROR,
    Qgis.Success: logging.INFO,
}

def forward_qgis_messages(message, tag, level):
    logging.getLogger(f"qgis.{tag or 'general'}").log(
        LEVELS.get(level, logging.INFO), message.strip()
    )

QgsApplication.messageLog().messageReceived.connect(forward_qgis_messages)

Breakdown: The signal fires for every message QGIS logs, with the originating tag — Processing, PostGIS, a plugin's name — which becomes the logger name so a noisy component can be filtered or silenced independently. Connecting this immediately after initQgis() catches provider errors that would otherwise be invisible, and it is often the single change that explains a mysterious failure: the answer was already being written, just not anywhere you could see.

Route Processing feedback into the same log

Every processing.run() accepts a feedback object. Supply one and the algorithm's progress and warnings become log lines rather than nothing.

from qgis.core import QgsProcessingFeedback

class LoggingFeedback(QgsProcessingFeedback):
    def __init__(self, logger):
        super().__init__(False)
        self._log = logger
        self._last = -1

    def pushInfo(self, info):
        self._log.info(info)

    def reportError(self, error, fatalError=False):
        self._log.error(error)

    def setProgress(self, progress):
        step = int(progress) // 10 * 10
        if step != self._last:
            self._last = step
            self._log.info("progress %d%%", step)

result = processing.run(
    "native:buffer",
    {"INPUT": source, "DISTANCE": 25, "OUTPUT": destination},
    feedback=LoggingFeedback(log),
)

Breakdown: super().__init__(False) disables the default console progress bar, which is meaningless in a log file. Overriding setProgress to log only when the tens digit changes turns a hundred updates per algorithm into ten lines — enough to see that a long run is advancing, few enough to read. reportError fires for recoverable problems as well as fatal ones, so a run can succeed while still having logged errors worth investigating. Passing the same feedback object through a chain of algorithms gives one continuous narrative, which pairs well with the patterns in Chaining Processing Algorithms in PyQGIS.

Assert that the result is plausible

Success is not "the code ran". Success is "the output looks like the thing it should look like". Check it explicitly.

EXPECTED_MINIMUM = 1000

written = QgsVectorLayer(destination, "output", "ogr")
if not written.isValid():
    log.error("output layer did not open: %s", destination)
    sys.exit(1)

count = written.featureCount()
log.info("wrote %d features to %s", count, destination)

if count < EXPECTED_MINIMUM:
    log.error("only %d features — expected at least %d, refusing to publish",
              count, EXPECTED_MINIMUM)
    sys.exit(2)

Breakdown: Re-opening the written file rather than trusting the writer's return value catches the case where the write succeeded and the file is unusable. The threshold check is the part that catches silent upstream breakage — a source feed that returns an empty response produces a technically successful run with zero features, and only a plausibility assertion turns that into an alert. Distinct exit codes let the scheduler treat "it crashed" and "it ran but the numbers are wrong" differently.

From finished run to exit codeA finished run is checked three times. If the output does not open, the job exits with code one. If it opens but the feature count is below the expected minimum, it exits with code two and does not publish. If both pass, the counts and duration are logged and the job exits zero.Exit codes are the only thing a scheduler readsrun finishedno exception raisedoutput opens?isValid()count plausible?above the floorexit 0log counts and durationexit 1broken outputexit 2suspicious result

Fail once, at the top

Wrap the whole job so that no exception can escape unlogged, and so the traceback lands in the same place as everything else.

def main():
    started = time.monotonic()
    log.info("starting nightly export")
    do_the_work()
    log.info("finished in %.1fs", time.monotonic() - started)
    return 0

if __name__ == "__main__":
    try:
        sys.exit(main())
    except Exception:
        log.exception("nightly export failed")
        sys.exit(1)
    finally:
        app.exitQgis()

Breakdown: log.exception() records the message together with the full traceback at ERROR level — the single most useful call in this file, and the reason not to catch exceptions deeper in the code where the context is thinner. time.monotonic() rather than time.time() is immune to clock adjustments, so a duration is never negative after an NTP correction. The finally block releases QGIS whether the run succeeded or not, which prevents a stuck process from holding database connections. Logging both a start and a finish line means an absent finish is itself a signal, and the locking pattern in the cron guide keeps a stuck run from being joined by another.

Retry the transient, fail the permanent

Some failures are worth trying again; most are not. Retrying a permanent error wastes time and buries the real message under three identical tracebacks, while not retrying a transient one turns a two-second network blip into a missed night.

import time

TRANSIENT = ("could not connect", "timeout expired", "connection reset", "temporarily unavailable")

def is_transient(message):
    lowered = message.lower()
    return any(marker in lowered for marker in TRANSIENT)

def with_retry(operation, attempts=3, delay=5):
    for attempt in range(1, attempts + 1):
        try:
            return operation()
        except Exception as exc:
            if attempt == attempts or not is_transient(str(exc)):
                raise
            log.warning("attempt %d failed (%s) — retrying in %ds", attempt, exc, delay * attempt)
            time.sleep(delay * attempt)

Breakdown: The classification is deliberately conservative: anything not recognised as transient is re-raised immediately, because a missing file, a syntax error in an expression or a permissions problem will fail identically on every attempt. Logging each failed attempt at warning level leaves a trail showing that the eventual success took three tries — information that disappears if the retry is silent, and that is exactly what tells you a link is degrading before it fails outright. The growing delay gives a restarting service time to come back.

Retry only operations that are safe to repeat. A read is; a write that may already have committed is not, which is why the database guidance pairs retries with a unique constraint and a transaction rather than with hope — see Append Features to a PostGIS Table in PyQGIS.

Which failures deserve a second attemptA failure is classified by its message. Connection timeouts and resets are treated as transient and retried with growing delays, succeeding on a later attempt or eventually failing with a clear history. A missing file, a permission error or a bad expression is permanent, so the job fails immediately rather than repeating the same error three times.Three identical tracebacks are not resiliencethe operation failedtransient — retryconnection reset · timeout expiredwait 5 s, then 10 s, logging each attemptpermanent — raise nowmissing file · permission denieda second attempt fails identicallyRetry reads freely; retry a write only when a unique constraint makes the repeat safe

QGIS version compatibility

The examples target QGIS 3.34 LTR (Python 3.12).

QGIS versionPythonNotes
3.22 LTR3.9messageReceived carries the same three arguments; Qgis.MessageLevel members are named identically.
3.28 LTR3.9QgsProcessingFeedback gains pushFormattedMessage; overriding pushInfo still captures everything.
3.34 LTR3.12Baseline for this page.
3.40 / 3.443.12reportError's fatalError argument is unchanged; new pushWarning may be overridden for finer control.

Troubleshooting

  • The log contains nothing from QGIS. messageReceived was connected after the failing operation, or the connection was made to a different application instance. Connect immediately after initQgis().
  • The traceback is missing from the log. log.error(str(exc)) was used instead of log.exception().
  • Log lines appear only when the job ends. Python is buffering standard output. Set PYTHONUNBUFFERED=1, or pass flush=True if you are printing.
  • Progress floods the log. The feedback object logs every update. Throttle it as shown, or drop progress below INFO level.
  • The job exits zero after an algorithm failed. processing.run() raises QgsProcessingException on failure, but only if you do not catch and ignore it; a bare except: pass anywhere in the chain hides it.
  • Timestamps disagree between machines. Log in UTC by setting logging.Formatter.converter = time.gmtime, and let the reader localise.

Conclusion

An unattended script needs four things wired together: Python logging configured to standard output, the QGIS message log forwarded into it, a feedback object passed to every algorithm, and a plausibility check on the result that can fail the run with a distinct exit code. Add a top-level handler that logs the traceback and always releases QGIS, and a silent failure stops being possible.

Frequently Asked Questions

Should I log to a file directly from Python? Prefer standard output and let the runner redirect it. That keeps one script working under cron, systemd and Docker, and it makes log rotation somebody else's job.

How verbose should a nightly job be? One line per stage plus counts, and progress at ten-percent intervals for anything slow. The test is whether two consecutive nights can be diffed usefully.

Can I send QGIS messages to syslog or the journal? Yes — attach a SysLogHandler or JournalHandler to the root logger. The forwarding shown above puts QGIS's messages into the same logging tree, so they inherit any handler you add.

Why not just use print()? Print has no levels, no timestamps and no way to be filtered or redirected per component. The moment you want to silence one noisy provider, you need logging.

How do I test the failure path? Point the job at an empty source and confirm it exits 2, and make it raise deliberately once to confirm the traceback lands in the log. A failure path that has never run is not a failure path.