[{"data":1,"prerenderedAt":2329},["ShallowReactive",2],{"doc:\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Frun-batch-jobs-from-csv-parameter-table-pyqgis":3},{"id":4,"title":5,"body":6,"description":2318,"extension":2319,"meta":2320,"navigation":319,"path":2325,"seo":2326,"stem":2327,"__hash__":2328},"docs\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Frun-batch-jobs-from-csv-parameter-table-pyqgis\u002Findex.md","Run Batch Jobs from a CSV Parameter Table in PyQGIS",{"type":7,"value":8,"toc":2304},"minimark",[9,13,17,26,153,158,168,172,175,185,204,208,211,268,956,965,969,972,1161,1170,1174,1177,1682,1691,1695,1698,1747,2000,2009,2013,2016,2169,2174,2178,2185,2189,2217,2221,2224,2228,2238,2248,2254,2260,2266,2270,2300],[10,11,5],"h1",{"id":12},"run-batch-jobs-from-a-csv-parameter-table-in-pyqgis",[14,15,16],"p",{},"Some batch jobs vary in more than their input file. Buffer each protected area by its own legally defined distance; clip each municipality's data to its boundary and write it to its own folder; run a suitability model with different weights for each scenario. The cleanest way to express such jobs is a table: one row per job, one column per parameter. Planners and analysts can maintain the table in a spreadsheet, and a short script runs whatever it describes — validating every row first, so a typo in row 47 does not fail an hour into the run.",[14,18,19,20,25],{},"This recipe belongs to ",[21,22,24],"a",{"href":23},"\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002F","Batch Processing with PyQGIS",". It designs a parameter table, reads and validates it, converts text values to the types Processing expects, runs the jobs, writes status and outputs back into a results table, and supports dry runs.",[14,27,28],{},[29,30,35,39,43,50,67,76,85,91,96,100,104,111,116,120,123,126,129,133,139,144,147,150],"svg",{"viewBox":31,"role":32,"ariaLabel":33,"xmlns":34},"0 0 760 236","img","A CSV parameter table validated in full, then run row by row with status and outputs recorded in a results table","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg",[36,37,38],"title",{},"A table of jobs",[40,41,42],"desc",{},"A CSV file with one row per job: an id, the algorithm, the input path, parameters such as distance, and the output path. The script validates every row first — files exist, algorithm exists, values parse — and reports all problems together. Valid rows run one by one, and a results table records status, feature count, duration and any error per job.",[44,45],"rect",{"x":46,"y":46,"width":47,"height":48,"fill":49},"0","760","236","#f6f3ea",[51,52,53],"defs",{},[54,55,62],"marker",{"id":56,"viewBox":57,"refX":58,"refY":59,"markerWidth":60,"markerHeight":60,"orient":61},"csvFlowArrow","0 0 10 10","8","5","7","auto-start-reverse",[63,64],"path",{"d":65,"fill":66},"M0 0 L10 5 L0 10 z","#2f3b35",[68,69,75],"text",{"x":70,"y":71,"style":72,"fill":73,"textAnchor":74},"380","28","text-anchor:middle;font-size:14px;font-family:sans-serif;font-weight:bold","#17211d","middle","Rows in, validated, run, recorded",[44,77],{"x":78,"y":79,"width":80,"height":81,"rx":58,"fill":82,"stroke":83,"style":84},"24","60","200","160","#fffdf7","#59645f","stroke-width:2",[68,86,90],{"x":87,"y":88,"style":89,"fill":73,"textAnchor":74},"124","101.78","text-anchor:middle;font-size:11.5px;font-family:sans-serif;font-weight:bold","jobs.csv",[68,92,95],{"x":87,"y":93,"style":94,"fill":66,"textAnchor":74},"129.78","text-anchor:middle;font-size:10.5px;font-family:sans-serif","id · algorithm",[68,97,99],{"x":87,"y":98,"style":94,"fill":66,"textAnchor":74},"157.78","input · distance",[68,101,103],{"x":87,"y":102,"style":94,"fill":66,"textAnchor":74},"185.78","output",[105,106],"line",{"x1":107,"y1":108,"x2":109,"y2":108,"stroke":66,"style":110},"224","140","266","stroke-width:1.8;marker-end:url(#csvFlowArrow)",[44,112],{"x":113,"y":79,"width":80,"height":81,"rx":58,"fill":114,"stroke":115,"style":84},"270","#fdf2e2","#b45309",[68,117,119],{"x":118,"y":88,"style":89,"fill":115,"textAnchor":74},"370","validate all",[68,121,122],{"x":118,"y":93,"style":94,"fill":66,"textAnchor":74},"files exist?",[68,124,125],{"x":118,"y":98,"style":94,"fill":66,"textAnchor":74},"values parse?",[68,127,128],{"x":118,"y":102,"style":94,"fill":83,"textAnchor":74},"report all errors",[105,130],{"x1":131,"y1":108,"x2":132,"y2":108,"stroke":66,"style":110},"470","512",[44,134],{"x":135,"y":79,"width":136,"height":81,"rx":58,"fill":137,"stroke":138,"style":84},"516","220","#e8efe6","#15803d",[68,140,143],{"x":141,"y":88,"style":89,"fill":142,"textAnchor":74},"626","#166534","run + record",[68,145,146],{"x":141,"y":93,"style":94,"fill":66,"textAnchor":74},"status · count",[68,148,149],{"x":141,"y":98,"style":94,"fill":66,"textAnchor":74},"duration · error",[68,151,152],{"x":141,"y":102,"style":94,"fill":83,"textAnchor":74},"results.csv",[154,155,157],"h2",{"id":156},"prerequisites","Prerequisites",[159,160,161,165],"ul",{},[162,163,164],"li",{},"QGIS 3.34 LTR or newer, or the QGIS 4 series.",[162,166,167],{},"A CSV of jobs saved as UTF-8. Spreadsheets export it with \"CSV UTF-8\"; check the delimiter, which is a semicolon in many European locales.",[154,169,171],{"id":170},"design-the-parameter-table","Design the parameter table",[14,173,174],{},"One row per job, one column per parameter, and a few columns for bookkeeping. Column names should be readable by the people maintaining the table.",[176,177,182],"pre",{"className":178,"code":180,"language":68,"meta":181},[179],"language-text","job_id;enabled;algorithm;input;distance_m;dissolve;output\nPA-001;1;native:buffer;\u002Fdata\u002Fprotected\u002Fpa_001.gpkg;250;1;\u002Fdata\u002Fout\u002Fpa_001_buffer.gpkg\nPA-002;1;native:buffer;\u002Fdata\u002Fprotected\u002Fpa_002.gpkg;500;1;\u002Fdata\u002Fout\u002Fpa_002_buffer.gpkg\nPA-003;0;native:buffer;\u002Fdata\u002Fprotected\u002Fpa_003.gpkg;250;0;\u002Fdata\u002Fout\u002Fpa_003_buffer.gpkg\n","",[183,184,180],"code",{"__ignoreMap":181},[14,186,187,191,192,195,196,199,200,203],{},[188,189,190],"strong",{},"Breakdown:"," A stable ",[183,193,194],{},"job_id"," identifies each row in reports and outputs. An ",[183,197,198],{},"enabled"," column lets maintainers switch jobs off without deleting rows. Units in column names (",[183,201,202],{},"distance_m",") prevent the classic mistake of entering kilometres. Keeping the algorithm in a column makes a table that mixes several kinds of job possible, though tables are easier to validate when every row uses the same algorithm.",[154,205,207],{"id":206},"read-and-validate-every-row-first","Read and validate every row first",[14,209,210],{},"Validation runs over the whole table before any processing, collecting every problem so they can be fixed in one pass.",[14,212,213],{},[29,214,217,220,223,226,229,235,240,244,248,252,255,259,262,265],{"viewBox":215,"role":32,"ariaLabel":216,"xmlns":34},"0 0 760 222","Validating all rows before running collects every problem at once, instead of failing midway through a run",[36,218,219],{},"Validate the whole table, then run",[40,221,222],{},"Validating row by row while running stops at the first bad row after earlier rows have already run. Validating the whole table first collects every problem — a missing file in row 12, a non-numeric distance in row 30, an unknown algorithm in row 41 — reports them all, and runs nothing until the table is clean.",[44,224],{"x":46,"y":46,"width":47,"height":225,"fill":49},"222",[68,227,228],{"x":70,"y":71,"style":72,"fill":73,"textAnchor":74},"Fail before running, not during",[44,230],{"x":78,"y":231,"width":232,"height":233,"rx":58,"fill":114,"stroke":234,"style":84},"56","340","150","#b91c1c",[68,236,239],{"x":237,"y":238,"style":89,"fill":234,"textAnchor":74},"194","92.78","validate while running",[68,241,243],{"x":237,"y":242,"style":94,"fill":66,"textAnchor":74},"120.78","row 12 fails after",[68,245,247],{"x":237,"y":246,"style":94,"fill":66,"textAnchor":74},"148.78","11 jobs already ran",[68,249,251],{"x":237,"y":250,"style":94,"fill":83,"textAnchor":74},"176.78","partial results",[44,253],{"x":254,"y":231,"width":232,"height":233,"rx":58,"fill":137,"stroke":138,"style":84},"396",[68,256,258],{"x":257,"y":238,"style":89,"fill":142,"textAnchor":74},"566","validate first",[68,260,261],{"x":257,"y":242,"style":94,"fill":66,"textAnchor":74},"all problems listed",[68,263,264],{"x":257,"y":246,"style":94,"fill":66,"textAnchor":74},"nothing ran yet",[68,266,267],{"x":257,"y":250,"style":94,"fill":83,"textAnchor":74},"fix, then run",[176,269,273],{"className":270,"code":271,"language":272,"meta":181,"style":181},"language-python shiki shiki-themes github-dark","import csv\nfrom pathlib import Path\nfrom qgis.core import QgsApplication\n\nregistry = QgsApplication.processingRegistry()\n\ndef load_jobs(path, delimiter=\";\"):\n    with open(path, newline=\"\", encoding=\"utf-8-sig\") as fh:\n        return list(csv.DictReader(fh, delimiter=delimiter))\n\ndef validate(jobs):\n    problems, seen = [], set()\n    for n, job in enumerate(jobs, start=2):           # row 1 is the header\n        jid = job.get(\"job_id\", \"\").strip()\n        if not jid or jid in seen:\n            problems.append(f\"row {n}: missing or duplicate job_id '{jid}'\")\n        seen.add(jid)\n        if job.get(\"enabled\", \"1\").strip() not in (\"0\", \"1\"):\n            problems.append(f\"row {n}: enabled must be 0 or 1\")\n        if registry.algorithmById(job.get(\"algorithm\", \"\")) is None:\n            problems.append(f\"row {n}: unknown algorithm '{job.get('algorithm')}'\")\n        if not Path(job.get(\"input\", \"\")).exists():\n            problems.append(f\"row {n}: input not found: {job.get('input')}\")\n        try:\n            if float(job.get(\"distance_m\", \"\").replace(\",\", \".\")) \u003C= 0:\n                problems.append(f\"row {n}: distance_m must be positive\")\n        except ValueError:\n            problems.append(f\"row {n}: distance_m is not a number: '{job.get('distance_m')}'\")\n    return problems\n\njobs = load_jobs(\"\u002Fdata\u002Fjobs\u002Fbuffers.csv\")\nproblems = validate(jobs)\nif problems:\n    print(\"\\n\".join(problems))\n    raise SystemExit(f\"{len(problems)} problems — nothing was run\")\n","python",[183,274,275,287,301,314,321,333,338,360,402,422,427,438,455,488,509,531,568,574,610,630,657,692,712,746,754,794,815,826,859,868,873,889,900,909,928],{"__ignoreMap":181},[276,277,279,283],"span",{"class":105,"line":278},1,[276,280,282],{"class":281},"snl16","import",[276,284,286],{"class":285},"s95oV"," csv\n",[276,288,290,293,296,298],{"class":105,"line":289},2,[276,291,292],{"class":281},"from",[276,294,295],{"class":285}," pathlib ",[276,297,282],{"class":281},[276,299,300],{"class":285}," Path\n",[276,302,304,306,309,311],{"class":105,"line":303},3,[276,305,292],{"class":281},[276,307,308],{"class":285}," qgis.core ",[276,310,282],{"class":281},[276,312,313],{"class":285}," QgsApplication\n",[276,315,317],{"class":105,"line":316},4,[276,318,320],{"emptyLinePlaceholder":319},true,"\n",[276,322,324,327,330],{"class":105,"line":323},5,[276,325,326],{"class":285},"registry ",[276,328,329],{"class":281},"=",[276,331,332],{"class":285}," QgsApplication.processingRegistry()\n",[276,334,336],{"class":105,"line":335},6,[276,337,320],{"emptyLinePlaceholder":319},[276,339,341,344,348,351,353,357],{"class":105,"line":340},7,[276,342,343],{"class":281},"def",[276,345,347],{"class":346},"svObZ"," load_jobs",[276,349,350],{"class":285},"(path, delimiter",[276,352,329],{"class":281},[276,354,356],{"class":355},"sU2Wk","\";\"",[276,358,359],{"class":285},"):\n",[276,361,363,366,370,373,377,379,382,385,388,390,393,396,399],{"class":105,"line":362},8,[276,364,365],{"class":281},"    with",[276,367,369],{"class":368},"sDLfK"," open",[276,371,372],{"class":285},"(path, ",[276,374,376],{"class":375},"s9osk","newline",[276,378,329],{"class":281},[276,380,381],{"class":355},"\"\"",[276,383,384],{"class":285},", ",[276,386,387],{"class":375},"encoding",[276,389,329],{"class":281},[276,391,392],{"class":355},"\"utf-8-sig\"",[276,394,395],{"class":285},") ",[276,397,398],{"class":281},"as",[276,400,401],{"class":285}," fh:\n",[276,403,405,408,411,414,417,419],{"class":105,"line":404},9,[276,406,407],{"class":281},"        return",[276,409,410],{"class":368}," list",[276,412,413],{"class":285},"(csv.DictReader(fh, ",[276,415,416],{"class":375},"delimiter",[276,418,329],{"class":281},[276,420,421],{"class":285},"delimiter))\n",[276,423,425],{"class":105,"line":424},10,[276,426,320],{"emptyLinePlaceholder":319},[276,428,430,432,435],{"class":105,"line":429},11,[276,431,343],{"class":281},[276,433,434],{"class":346}," validate",[276,436,437],{"class":285},"(jobs):\n",[276,439,441,444,446,449,452],{"class":105,"line":440},12,[276,442,443],{"class":285},"    problems, seen ",[276,445,329],{"class":281},[276,447,448],{"class":285}," [], ",[276,450,451],{"class":368},"set",[276,453,454],{"class":285},"()\n",[276,456,458,461,464,467,470,473,476,478,481,484],{"class":105,"line":457},13,[276,459,460],{"class":281},"    for",[276,462,463],{"class":285}," n, job ",[276,465,466],{"class":281},"in",[276,468,469],{"class":368}," enumerate",[276,471,472],{"class":285},"(jobs, ",[276,474,475],{"class":375},"start",[276,477,329],{"class":281},[276,479,480],{"class":368},"2",[276,482,483],{"class":285},"):           ",[276,485,487],{"class":486},"sjoCn","# row 1 is the header\n",[276,489,491,494,496,499,502,504,506],{"class":105,"line":490},14,[276,492,493],{"class":285},"        jid ",[276,495,329],{"class":281},[276,497,498],{"class":285}," job.get(",[276,500,501],{"class":355},"\"job_id\"",[276,503,384],{"class":285},[276,505,381],{"class":355},[276,507,508],{"class":285},").strip()\n",[276,510,512,515,518,521,524,526,528],{"class":105,"line":511},15,[276,513,514],{"class":281},"        if",[276,516,517],{"class":281}," not",[276,519,520],{"class":285}," jid ",[276,522,523],{"class":281},"or",[276,525,520],{"class":285},[276,527,466],{"class":281},[276,529,530],{"class":285}," seen:\n",[276,532,534,537,540,543,546,549,552,555,557,560,562,565],{"class":105,"line":533},16,[276,535,536],{"class":285},"            problems.append(",[276,538,539],{"class":281},"f",[276,541,542],{"class":355},"\"row ",[276,544,545],{"class":368},"{",[276,547,548],{"class":285},"n",[276,550,551],{"class":368},"}",[276,553,554],{"class":355},": missing or duplicate job_id '",[276,556,545],{"class":368},[276,558,559],{"class":285},"jid",[276,561,551],{"class":368},[276,563,564],{"class":355},"'\"",[276,566,567],{"class":285},")\n",[276,569,571],{"class":105,"line":570},17,[276,572,573],{"class":285},"        seen.add(jid)\n",[276,575,577,579,581,584,586,589,592,595,598,601,604,606,608],{"class":105,"line":576},18,[276,578,514],{"class":281},[276,580,498],{"class":285},[276,582,583],{"class":355},"\"enabled\"",[276,585,384],{"class":285},[276,587,588],{"class":355},"\"1\"",[276,590,591],{"class":285},").strip() ",[276,593,594],{"class":281},"not",[276,596,597],{"class":281}," in",[276,599,600],{"class":285}," (",[276,602,603],{"class":355},"\"0\"",[276,605,384],{"class":285},[276,607,588],{"class":355},[276,609,359],{"class":285},[276,611,613,615,617,619,621,623,625,628],{"class":105,"line":612},19,[276,614,536],{"class":285},[276,616,539],{"class":281},[276,618,542],{"class":355},[276,620,545],{"class":368},[276,622,548],{"class":285},[276,624,551],{"class":368},[276,626,627],{"class":355},": enabled must be 0 or 1\"",[276,629,567],{"class":285},[276,631,633,635,638,641,643,645,648,651,654],{"class":105,"line":632},20,[276,634,514],{"class":281},[276,636,637],{"class":285}," registry.algorithmById(job.get(",[276,639,640],{"class":355},"\"algorithm\"",[276,642,384],{"class":285},[276,644,381],{"class":355},[276,646,647],{"class":285},")) ",[276,649,650],{"class":281},"is",[276,652,653],{"class":368}," None",[276,655,656],{"class":285},":\n",[276,658,660,662,664,666,668,670,672,675,677,680,683,686,688,690],{"class":105,"line":659},21,[276,661,536],{"class":285},[276,663,539],{"class":281},[276,665,542],{"class":355},[276,667,545],{"class":368},[276,669,548],{"class":285},[276,671,551],{"class":368},[276,673,674],{"class":355},": unknown algorithm '",[276,676,545],{"class":368},[276,678,679],{"class":285},"job.get(",[276,681,682],{"class":355},"'algorithm'",[276,684,685],{"class":285},")",[276,687,551],{"class":368},[276,689,564],{"class":355},[276,691,567],{"class":285},[276,693,695,697,699,702,705,707,709],{"class":105,"line":694},22,[276,696,514],{"class":281},[276,698,517],{"class":281},[276,700,701],{"class":285}," Path(job.get(",[276,703,704],{"class":355},"\"input\"",[276,706,384],{"class":285},[276,708,381],{"class":355},[276,710,711],{"class":285},")).exists():\n",[276,713,715,717,719,721,723,725,727,730,732,734,737,739,741,744],{"class":105,"line":714},23,[276,716,536],{"class":285},[276,718,539],{"class":281},[276,720,542],{"class":355},[276,722,545],{"class":368},[276,724,548],{"class":285},[276,726,551],{"class":368},[276,728,729],{"class":355},": input not found: ",[276,731,545],{"class":368},[276,733,679],{"class":285},[276,735,736],{"class":355},"'input'",[276,738,685],{"class":285},[276,740,551],{"class":368},[276,742,743],{"class":355},"\"",[276,745,567],{"class":285},[276,747,749,752],{"class":105,"line":748},24,[276,750,751],{"class":281},"        try",[276,753,656],{"class":285},[276,755,757,760,763,766,769,771,773,776,779,781,784,786,789,792],{"class":105,"line":756},25,[276,758,759],{"class":281},"            if",[276,761,762],{"class":368}," float",[276,764,765],{"class":285},"(job.get(",[276,767,768],{"class":355},"\"distance_m\"",[276,770,384],{"class":285},[276,772,381],{"class":355},[276,774,775],{"class":285},").replace(",[276,777,778],{"class":355},"\",\"",[276,780,384],{"class":285},[276,782,783],{"class":355},"\".\"",[276,785,647],{"class":285},[276,787,788],{"class":281},"\u003C=",[276,790,791],{"class":368}," 0",[276,793,656],{"class":285},[276,795,797,800,802,804,806,808,810,813],{"class":105,"line":796},26,[276,798,799],{"class":285},"                problems.append(",[276,801,539],{"class":281},[276,803,542],{"class":355},[276,805,545],{"class":368},[276,807,548],{"class":285},[276,809,551],{"class":368},[276,811,812],{"class":355},": distance_m must be positive\"",[276,814,567],{"class":285},[276,816,818,821,824],{"class":105,"line":817},27,[276,819,820],{"class":281},"        except",[276,822,823],{"class":368}," ValueError",[276,825,656],{"class":285},[276,827,829,831,833,835,837,839,841,844,846,848,851,853,855,857],{"class":105,"line":828},28,[276,830,536],{"class":285},[276,832,539],{"class":281},[276,834,542],{"class":355},[276,836,545],{"class":368},[276,838,548],{"class":285},[276,840,551],{"class":368},[276,842,843],{"class":355},": distance_m is not a number: '",[276,845,545],{"class":368},[276,847,679],{"class":285},[276,849,850],{"class":355},"'distance_m'",[276,852,685],{"class":285},[276,854,551],{"class":368},[276,856,564],{"class":355},[276,858,567],{"class":285},[276,860,862,865],{"class":105,"line":861},29,[276,863,864],{"class":281},"    return",[276,866,867],{"class":285}," problems\n",[276,869,871],{"class":105,"line":870},30,[276,872,320],{"emptyLinePlaceholder":319},[276,874,876,879,881,884,887],{"class":105,"line":875},31,[276,877,878],{"class":285},"jobs ",[276,880,329],{"class":281},[276,882,883],{"class":285}," load_jobs(",[276,885,886],{"class":355},"\"\u002Fdata\u002Fjobs\u002Fbuffers.csv\"",[276,888,567],{"class":285},[276,890,892,895,897],{"class":105,"line":891},32,[276,893,894],{"class":285},"problems ",[276,896,329],{"class":281},[276,898,899],{"class":285}," validate(jobs)\n",[276,901,903,906],{"class":105,"line":902},33,[276,904,905],{"class":281},"if",[276,907,908],{"class":285}," problems:\n",[276,910,912,915,918,920,923,925],{"class":105,"line":911},34,[276,913,914],{"class":368},"    print",[276,916,917],{"class":285},"(",[276,919,743],{"class":355},[276,921,922],{"class":368},"\\n",[276,924,743],{"class":355},[276,926,927],{"class":285},".join(problems))\n",[276,929,931,934,937,939,941,943,946,949,951,954],{"class":105,"line":930},35,[276,932,933],{"class":281},"    raise",[276,935,936],{"class":368}," SystemExit",[276,938,917],{"class":285},[276,940,539],{"class":281},[276,942,743],{"class":355},[276,944,945],{"class":368},"{len",[276,947,948],{"class":285},"(problems)",[276,950,551],{"class":368},[276,952,953],{"class":355}," problems — nothing was run\"",[276,955,567],{"class":285},[14,957,958,960,961,964],{},[188,959,190],{}," ",[183,962,963],{},"utf-8-sig"," handles the byte-order mark some spreadsheet exports add, which otherwise corrupts the first column name. Row numbers in messages refer to spreadsheet rows, header included, so maintainers can jump straight to the problem. Every rule appends a message instead of raising, so one run of the validator reports every issue. Accepting a decimal comma in numbers spares European users a confusing error. Nothing runs until the table is clean.",[154,966,968],{"id":967},"convert-text-to-the-types-algorithms-expect","Convert text to the types algorithms expect",[14,970,971],{},"CSV values are strings; Processing parameters need numbers, booleans, enums and paths. A small conversion layer keeps the job loop simple.",[176,973,975],{"className":270,"code":974,"language":272,"meta":181,"style":181},"def to_bool(text):\n    return text.strip().lower() in (\"1\", \"true\", \"yes\", \"y\")\n\ndef to_float(text):\n    return float(text.strip().replace(\",\", \".\"))\n\ndef params_for(job):\n    return {\n        \"INPUT\": job[\"input\"],\n        \"DISTANCE\": to_float(job[\"distance_m\"]),\n        \"SEGMENTS\": 16,\n        \"DISSOLVE\": to_bool(job.get(\"dissolve\", \"0\")),\n        \"OUTPUT\": job[\"output\"],\n    }\n\nprint(params_for(jobs[0]))\n",[183,976,977,987,1017,1021,1030,1048,1052,1062,1069,1082,1095,1109,1127,1139,1144,1148],{"__ignoreMap":181},[276,978,979,981,984],{"class":105,"line":278},[276,980,343],{"class":281},[276,982,983],{"class":346}," to_bool",[276,985,986],{"class":285},"(text):\n",[276,988,989,991,994,996,998,1000,1002,1005,1007,1010,1012,1015],{"class":105,"line":289},[276,990,864],{"class":281},[276,992,993],{"class":285}," text.strip().lower() ",[276,995,466],{"class":281},[276,997,600],{"class":285},[276,999,588],{"class":355},[276,1001,384],{"class":285},[276,1003,1004],{"class":355},"\"true\"",[276,1006,384],{"class":285},[276,1008,1009],{"class":355},"\"yes\"",[276,1011,384],{"class":285},[276,1013,1014],{"class":355},"\"y\"",[276,1016,567],{"class":285},[276,1018,1019],{"class":105,"line":303},[276,1020,320],{"emptyLinePlaceholder":319},[276,1022,1023,1025,1028],{"class":105,"line":316},[276,1024,343],{"class":281},[276,1026,1027],{"class":346}," to_float",[276,1029,986],{"class":285},[276,1031,1032,1034,1036,1039,1041,1043,1045],{"class":105,"line":323},[276,1033,864],{"class":281},[276,1035,762],{"class":368},[276,1037,1038],{"class":285},"(text.strip().replace(",[276,1040,778],{"class":355},[276,1042,384],{"class":285},[276,1044,783],{"class":355},[276,1046,1047],{"class":285},"))\n",[276,1049,1050],{"class":105,"line":335},[276,1051,320],{"emptyLinePlaceholder":319},[276,1053,1054,1056,1059],{"class":105,"line":340},[276,1055,343],{"class":281},[276,1057,1058],{"class":346}," params_for",[276,1060,1061],{"class":285},"(job):\n",[276,1063,1064,1066],{"class":105,"line":362},[276,1065,864],{"class":281},[276,1067,1068],{"class":285}," {\n",[276,1070,1071,1074,1077,1079],{"class":105,"line":404},[276,1072,1073],{"class":355},"        \"INPUT\"",[276,1075,1076],{"class":285},": job[",[276,1078,704],{"class":355},[276,1080,1081],{"class":285},"],\n",[276,1083,1084,1087,1090,1092],{"class":105,"line":424},[276,1085,1086],{"class":355},"        \"DISTANCE\"",[276,1088,1089],{"class":285},": to_float(job[",[276,1091,768],{"class":355},[276,1093,1094],{"class":285},"]),\n",[276,1096,1097,1100,1103,1106],{"class":105,"line":429},[276,1098,1099],{"class":355},"        \"SEGMENTS\"",[276,1101,1102],{"class":285},": ",[276,1104,1105],{"class":368},"16",[276,1107,1108],{"class":285},",\n",[276,1110,1111,1114,1117,1120,1122,1124],{"class":105,"line":440},[276,1112,1113],{"class":355},"        \"DISSOLVE\"",[276,1115,1116],{"class":285},": to_bool(job.get(",[276,1118,1119],{"class":355},"\"dissolve\"",[276,1121,384],{"class":285},[276,1123,603],{"class":355},[276,1125,1126],{"class":285},")),\n",[276,1128,1129,1132,1134,1137],{"class":105,"line":457},[276,1130,1131],{"class":355},"        \"OUTPUT\"",[276,1133,1076],{"class":285},[276,1135,1136],{"class":355},"\"output\"",[276,1138,1081],{"class":285},[276,1140,1141],{"class":105,"line":490},[276,1142,1143],{"class":285},"    }\n",[276,1145,1146],{"class":105,"line":511},[276,1147,320],{"emptyLinePlaceholder":319},[276,1149,1150,1153,1156,1158],{"class":105,"line":533},[276,1151,1152],{"class":368},"print",[276,1154,1155],{"class":285},"(params_for(jobs[",[276,1157,46],{"class":368},[276,1159,1160],{"class":285},"]))\n",[14,1162,1163,1165,1166,1169],{},[188,1164,190],{}," Explicit converters make the mapping from table to algorithm visible and testable. Values not in the table — segments here — are set once in code, so the table only contains what actually varies. For algorithms with enum parameters, map readable words in the table (\"round\", \"flat\") to the enum numbers in the converter rather than asking maintainers to type numbers. Check parameter names with ",[183,1167,1168],{},"processing.algorithmHelp",".",[154,1171,1173],{"id":1172},"run-the-jobs-and-record-results","Run the jobs and record results",[14,1175,1176],{},"Each enabled row runs once; status, feature count, duration and any error are written to a results table that mirrors the job table.",[176,1178,1180],{"className":270,"code":1179,"language":272,"meta":181,"style":181},"import time\nimport processing\nfrom datetime import datetime\nfrom qgis.core import QgsVectorLayer\n\nresults = []\nfor job in jobs:\n    if job[\"enabled\"].strip() != \"1\":\n        results.append({**job, \"status\": \"disabled\"})\n        continue\n    Path(job[\"output\"]).parent.mkdir(parents=True, exist_ok=True)\n    start = time.perf_counter()\n    try:\n        out = processing.run(job[\"algorithm\"], params_for(job))[\"OUTPUT\"]\n        count = QgsVectorLayer(out, \"check\", \"ogr\").featureCount()\n        status, error = \"ok\", \"\"\n    except Exception as err:\n        count, status, error = 0, \"failed\", str(err)[:300]\n    results.append({**job, \"status\": status, \"features\": count,\n                    \"seconds\": round(time.perf_counter() - start, 1), \"error\": error,\n                    \"run_at\": datetime.now().isoformat(timespec=\"seconds\")})\n    print(f\"{job['job_id']:\u003C8} {status:\u003C8} {count:>7}\")\n\nwith open(\"\u002Fdata\u002Fjobs\u002Fbuffers_results.csv\", \"w\", newline=\"\", encoding=\"utf-8\") as fh:\n    w = csv.DictWriter(fh, fieldnames=list(results[0].keys()) + [\"features\", \"seconds\", \"error\", \"run_at\"],\n                       delimiter=\";\", extrasaction=\"ignore\")\n    w.writeheader()\n    w.writerows(results)\n",[183,1181,1182,1189,1196,1208,1219,1223,1233,1246,1267,1289,1294,1323,1333,1340,1361,1382,1397,1411,1438,1458,1489,1508,1558,1562,1602,1651,1672,1677],{"__ignoreMap":181},[276,1183,1184,1186],{"class":105,"line":278},[276,1185,282],{"class":281},[276,1187,1188],{"class":285}," time\n",[276,1190,1191,1193],{"class":105,"line":289},[276,1192,282],{"class":281},[276,1194,1195],{"class":285}," processing\n",[276,1197,1198,1200,1203,1205],{"class":105,"line":303},[276,1199,292],{"class":281},[276,1201,1202],{"class":285}," datetime ",[276,1204,282],{"class":281},[276,1206,1207],{"class":285}," datetime\n",[276,1209,1210,1212,1214,1216],{"class":105,"line":316},[276,1211,292],{"class":281},[276,1213,308],{"class":285},[276,1215,282],{"class":281},[276,1217,1218],{"class":285}," QgsVectorLayer\n",[276,1220,1221],{"class":105,"line":323},[276,1222,320],{"emptyLinePlaceholder":319},[276,1224,1225,1228,1230],{"class":105,"line":335},[276,1226,1227],{"class":285},"results ",[276,1229,329],{"class":281},[276,1231,1232],{"class":285}," []\n",[276,1234,1235,1238,1241,1243],{"class":105,"line":340},[276,1236,1237],{"class":281},"for",[276,1239,1240],{"class":285}," job ",[276,1242,466],{"class":281},[276,1244,1245],{"class":285}," jobs:\n",[276,1247,1248,1251,1254,1256,1259,1262,1265],{"class":105,"line":362},[276,1249,1250],{"class":281},"    if",[276,1252,1253],{"class":285}," job[",[276,1255,583],{"class":355},[276,1257,1258],{"class":285},"].strip() ",[276,1260,1261],{"class":281},"!=",[276,1263,1264],{"class":355}," \"1\"",[276,1266,656],{"class":285},[276,1268,1269,1272,1275,1278,1281,1283,1286],{"class":105,"line":404},[276,1270,1271],{"class":285},"        results.append({",[276,1273,1274],{"class":281},"**",[276,1276,1277],{"class":285},"job, ",[276,1279,1280],{"class":355},"\"status\"",[276,1282,1102],{"class":285},[276,1284,1285],{"class":355},"\"disabled\"",[276,1287,1288],{"class":285},"})\n",[276,1290,1291],{"class":105,"line":424},[276,1292,1293],{"class":281},"        continue\n",[276,1295,1296,1299,1301,1304,1307,1309,1312,1314,1317,1319,1321],{"class":105,"line":429},[276,1297,1298],{"class":285},"    Path(job[",[276,1300,1136],{"class":355},[276,1302,1303],{"class":285},"]).parent.mkdir(",[276,1305,1306],{"class":375},"parents",[276,1308,329],{"class":281},[276,1310,1311],{"class":368},"True",[276,1313,384],{"class":285},[276,1315,1316],{"class":375},"exist_ok",[276,1318,329],{"class":281},[276,1320,1311],{"class":368},[276,1322,567],{"class":285},[276,1324,1325,1328,1330],{"class":105,"line":440},[276,1326,1327],{"class":285},"    start ",[276,1329,329],{"class":281},[276,1331,1332],{"class":285}," time.perf_counter()\n",[276,1334,1335,1338],{"class":105,"line":457},[276,1336,1337],{"class":281},"    try",[276,1339,656],{"class":285},[276,1341,1342,1345,1347,1350,1352,1355,1358],{"class":105,"line":490},[276,1343,1344],{"class":285},"        out ",[276,1346,329],{"class":281},[276,1348,1349],{"class":285}," processing.run(job[",[276,1351,640],{"class":355},[276,1353,1354],{"class":285},"], params_for(job))[",[276,1356,1357],{"class":355},"\"OUTPUT\"",[276,1359,1360],{"class":285},"]\n",[276,1362,1363,1366,1368,1371,1374,1376,1379],{"class":105,"line":511},[276,1364,1365],{"class":285},"        count ",[276,1367,329],{"class":281},[276,1369,1370],{"class":285}," QgsVectorLayer(out, ",[276,1372,1373],{"class":355},"\"check\"",[276,1375,384],{"class":285},[276,1377,1378],{"class":355},"\"ogr\"",[276,1380,1381],{"class":285},").featureCount()\n",[276,1383,1384,1387,1389,1392,1394],{"class":105,"line":533},[276,1385,1386],{"class":285},"        status, error ",[276,1388,329],{"class":281},[276,1390,1391],{"class":355}," \"ok\"",[276,1393,384],{"class":285},[276,1395,1396],{"class":355},"\"\"\n",[276,1398,1399,1402,1405,1408],{"class":105,"line":570},[276,1400,1401],{"class":281},"    except",[276,1403,1404],{"class":368}," Exception",[276,1406,1407],{"class":281}," as",[276,1409,1410],{"class":285}," err:\n",[276,1412,1413,1416,1418,1420,1422,1425,1427,1430,1433,1436],{"class":105,"line":576},[276,1414,1415],{"class":285},"        count, status, error ",[276,1417,329],{"class":281},[276,1419,791],{"class":368},[276,1421,384],{"class":285},[276,1423,1424],{"class":355},"\"failed\"",[276,1426,384],{"class":285},[276,1428,1429],{"class":368},"str",[276,1431,1432],{"class":285},"(err)[:",[276,1434,1435],{"class":368},"300",[276,1437,1360],{"class":285},[276,1439,1440,1443,1445,1447,1449,1452,1455],{"class":105,"line":612},[276,1441,1442],{"class":285},"    results.append({",[276,1444,1274],{"class":281},[276,1446,1277],{"class":285},[276,1448,1280],{"class":355},[276,1450,1451],{"class":285},": status, ",[276,1453,1454],{"class":355},"\"features\"",[276,1456,1457],{"class":285},": count,\n",[276,1459,1460,1463,1465,1468,1471,1474,1477,1480,1483,1486],{"class":105,"line":632},[276,1461,1462],{"class":355},"                    \"seconds\"",[276,1464,1102],{"class":285},[276,1466,1467],{"class":368},"round",[276,1469,1470],{"class":285},"(time.perf_counter() ",[276,1472,1473],{"class":281},"-",[276,1475,1476],{"class":285}," start, ",[276,1478,1479],{"class":368},"1",[276,1481,1482],{"class":285},"), ",[276,1484,1485],{"class":355},"\"error\"",[276,1487,1488],{"class":285},": error,\n",[276,1490,1491,1494,1497,1500,1502,1505],{"class":105,"line":659},[276,1492,1493],{"class":355},"                    \"run_at\"",[276,1495,1496],{"class":285},": datetime.now().isoformat(",[276,1498,1499],{"class":375},"timespec",[276,1501,329],{"class":281},[276,1503,1504],{"class":355},"\"seconds\"",[276,1506,1507],{"class":285},")})\n",[276,1509,1510,1512,1514,1516,1518,1520,1523,1526,1529,1532,1534,1537,1540,1542,1544,1546,1549,1552,1554,1556],{"class":105,"line":694},[276,1511,914],{"class":368},[276,1513,917],{"class":285},[276,1515,539],{"class":281},[276,1517,743],{"class":355},[276,1519,545],{"class":368},[276,1521,1522],{"class":285},"job[",[276,1524,1525],{"class":355},"'job_id'",[276,1527,1528],{"class":285},"]",[276,1530,1531],{"class":281},":\u003C8",[276,1533,551],{"class":368},[276,1535,1536],{"class":368}," {",[276,1538,1539],{"class":285},"status",[276,1541,1531],{"class":281},[276,1543,551],{"class":368},[276,1545,1536],{"class":368},[276,1547,1548],{"class":285},"count",[276,1550,1551],{"class":281},":>7",[276,1553,551],{"class":368},[276,1555,743],{"class":355},[276,1557,567],{"class":285},[276,1559,1560],{"class":105,"line":714},[276,1561,320],{"emptyLinePlaceholder":319},[276,1563,1564,1567,1569,1571,1574,1576,1579,1581,1583,1585,1587,1589,1591,1593,1596,1598,1600],{"class":105,"line":748},[276,1565,1566],{"class":281},"with",[276,1568,369],{"class":368},[276,1570,917],{"class":285},[276,1572,1573],{"class":355},"\"\u002Fdata\u002Fjobs\u002Fbuffers_results.csv\"",[276,1575,384],{"class":285},[276,1577,1578],{"class":355},"\"w\"",[276,1580,384],{"class":285},[276,1582,376],{"class":375},[276,1584,329],{"class":281},[276,1586,381],{"class":355},[276,1588,384],{"class":285},[276,1590,387],{"class":375},[276,1592,329],{"class":281},[276,1594,1595],{"class":355},"\"utf-8\"",[276,1597,395],{"class":285},[276,1599,398],{"class":281},[276,1601,401],{"class":285},[276,1603,1604,1607,1609,1612,1615,1617,1620,1623,1625,1628,1631,1634,1636,1638,1640,1642,1644,1646,1649],{"class":105,"line":756},[276,1605,1606],{"class":285},"    w ",[276,1608,329],{"class":281},[276,1610,1611],{"class":285}," csv.DictWriter(fh, ",[276,1613,1614],{"class":375},"fieldnames",[276,1616,329],{"class":281},[276,1618,1619],{"class":368},"list",[276,1621,1622],{"class":285},"(results[",[276,1624,46],{"class":368},[276,1626,1627],{"class":285},"].keys()) ",[276,1629,1630],{"class":281},"+",[276,1632,1633],{"class":285}," [",[276,1635,1454],{"class":355},[276,1637,384],{"class":285},[276,1639,1504],{"class":355},[276,1641,384],{"class":285},[276,1643,1485],{"class":355},[276,1645,384],{"class":285},[276,1647,1648],{"class":355},"\"run_at\"",[276,1650,1081],{"class":285},[276,1652,1653,1656,1658,1660,1662,1665,1667,1670],{"class":105,"line":796},[276,1654,1655],{"class":375},"                       delimiter",[276,1657,329],{"class":281},[276,1659,356],{"class":355},[276,1661,384],{"class":285},[276,1663,1664],{"class":375},"extrasaction",[276,1666,329],{"class":281},[276,1668,1669],{"class":355},"\"ignore\"",[276,1671,567],{"class":285},[276,1673,1674],{"class":105,"line":817},[276,1675,1676],{"class":285},"    w.writeheader()\n",[276,1678,1679],{"class":105,"line":828},[276,1680,1681],{"class":285},"    w.writerows(results)\n",[14,1683,1684,1686,1687,1169],{},[188,1685,190],{}," Copying every job column into its result row means the results table can be opened next to the job table in the same spreadsheet and read row for row. Disabled jobs appear with their status, so the results account for every row. Truncating error messages keeps cells readable. Opening each output to count features confirms that something was actually written — a successful run that produced an empty layer is worth noticing. For large tables, combine with ",[21,1688,1690],{"href":1689},"\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Fparallelise-batch-jobs-with-multiprocessing-pyqgis\u002F","parallelising batch jobs",[154,1692,1694],{"id":1693},"rerun-only-what-failed","Rerun only what failed",[14,1696,1697],{},"After fixing problems, rerunning everything wastes time. Reading the previous results and running only failed or new jobs makes the table a living to-do list.",[14,1699,1700],{},[29,1701,1704,1707,1710,1713,1716,1719,1724,1728,1731,1734,1737,1740,1744],{"viewBox":1702,"role":32,"ariaLabel":1703,"xmlns":34},"0 0 760 196","A rerun that skips jobs previously successful with unchanged parameters and runs failed, new or changed jobs",[36,1705,1706],{},"Rerun only what needs it",[40,1708,1709],{},"Before running, the previous results table is read. Jobs whose last status was ok and whose output still exists are skipped. Jobs that failed, are new, or whose parameters changed since the last run are run again. The table can be edited and rerun repeatedly until every job is ok.",[44,1711],{"x":46,"y":46,"width":47,"height":1712,"fill":49},"196",[68,1714,1715],{"x":70,"y":71,"style":72,"fill":73,"textAnchor":74},"Skip what is done and unchanged",[44,1717],{"x":78,"y":79,"width":225,"height":1718,"rx":58,"fill":137,"stroke":138,"style":84},"120",[68,1720,1723],{"x":1721,"y":1722,"style":89,"fill":142,"textAnchor":74},"135","110.78","ok + unchanged",[68,1725,1727],{"x":1721,"y":1726,"style":94,"fill":83,"textAnchor":74},"136.78","skip",[44,1729],{"x":1730,"y":79,"width":225,"height":1718,"rx":58,"fill":114,"stroke":234,"style":84},"269",[68,1732,1733],{"x":70,"y":1722,"style":89,"fill":234,"textAnchor":74},"failed",[68,1735,1736],{"x":70,"y":1726,"style":94,"fill":83,"textAnchor":74},"run again",[44,1738],{"x":1739,"y":79,"width":225,"height":1718,"rx":58,"fill":114,"stroke":115,"style":84},"514",[68,1741,1743],{"x":1742,"y":1722,"style":89,"fill":115,"textAnchor":74},"625","new or changed",[68,1745,1746],{"x":1742,"y":1726,"style":94,"fill":83,"textAnchor":74},"run",[176,1748,1750],{"className":270,"code":1749,"language":272,"meta":181,"style":181},"import hashlib\n\ndef job_hash(job):\n    keys = (\"algorithm\", \"input\", \"distance_m\", \"dissolve\", \"output\")\n    return hashlib.sha1(\"|\".join(job.get(k, \"\") for k in keys).encode()).hexdigest()\n\nprevious = {}\nprev_path = Path(\"\u002Fdata\u002Fjobs\u002Fbuffers_results.csv\")\nif prev_path.exists():\n    for r in load_jobs(prev_path):\n        previous[r[\"job_id\"]] = r\n\ntodo = [j for j in jobs\n        if not (previous.get(j[\"job_id\"], {}).get(\"status\") == \"ok\"\n                and previous[j[\"job_id\"]].get(\"hash\") == job_hash(j)\n                and Path(j[\"output\"]).exists())]\nprint(len(todo), \"of\", len(jobs), \"jobs need running\")\n",[183,1751,1752,1759,1763,1772,1801,1828,1832,1842,1856,1863,1875,1890,1894,1914,1938,1961,1973],{"__ignoreMap":181},[276,1753,1754,1756],{"class":105,"line":278},[276,1755,282],{"class":281},[276,1757,1758],{"class":285}," hashlib\n",[276,1760,1761],{"class":105,"line":289},[276,1762,320],{"emptyLinePlaceholder":319},[276,1764,1765,1767,1770],{"class":105,"line":303},[276,1766,343],{"class":281},[276,1768,1769],{"class":346}," job_hash",[276,1771,1061],{"class":285},[276,1773,1774,1777,1779,1781,1783,1785,1787,1789,1791,1793,1795,1797,1799],{"class":105,"line":316},[276,1775,1776],{"class":285},"    keys ",[276,1778,329],{"class":281},[276,1780,600],{"class":285},[276,1782,640],{"class":355},[276,1784,384],{"class":285},[276,1786,704],{"class":355},[276,1788,384],{"class":285},[276,1790,768],{"class":355},[276,1792,384],{"class":285},[276,1794,1119],{"class":355},[276,1796,384],{"class":285},[276,1798,1136],{"class":355},[276,1800,567],{"class":285},[276,1802,1803,1805,1808,1811,1814,1816,1818,1820,1823,1825],{"class":105,"line":323},[276,1804,864],{"class":281},[276,1806,1807],{"class":285}," hashlib.sha1(",[276,1809,1810],{"class":355},"\"|\"",[276,1812,1813],{"class":285},".join(job.get(k, ",[276,1815,381],{"class":355},[276,1817,395],{"class":285},[276,1819,1237],{"class":281},[276,1821,1822],{"class":285}," k ",[276,1824,466],{"class":281},[276,1826,1827],{"class":285}," keys).encode()).hexdigest()\n",[276,1829,1830],{"class":105,"line":335},[276,1831,320],{"emptyLinePlaceholder":319},[276,1833,1834,1837,1839],{"class":105,"line":340},[276,1835,1836],{"class":285},"previous ",[276,1838,329],{"class":281},[276,1840,1841],{"class":285}," {}\n",[276,1843,1844,1847,1849,1852,1854],{"class":105,"line":362},[276,1845,1846],{"class":285},"prev_path ",[276,1848,329],{"class":281},[276,1850,1851],{"class":285}," Path(",[276,1853,1573],{"class":355},[276,1855,567],{"class":285},[276,1857,1858,1860],{"class":105,"line":404},[276,1859,905],{"class":281},[276,1861,1862],{"class":285}," prev_path.exists():\n",[276,1864,1865,1867,1870,1872],{"class":105,"line":424},[276,1866,460],{"class":281},[276,1868,1869],{"class":285}," r ",[276,1871,466],{"class":281},[276,1873,1874],{"class":285}," load_jobs(prev_path):\n",[276,1876,1877,1880,1882,1885,1887],{"class":105,"line":429},[276,1878,1879],{"class":285},"        previous[r[",[276,1881,501],{"class":355},[276,1883,1884],{"class":285},"]] ",[276,1886,329],{"class":281},[276,1888,1889],{"class":285}," r\n",[276,1891,1892],{"class":105,"line":440},[276,1893,320],{"emptyLinePlaceholder":319},[276,1895,1896,1899,1901,1904,1906,1909,1911],{"class":105,"line":457},[276,1897,1898],{"class":285},"todo ",[276,1900,329],{"class":281},[276,1902,1903],{"class":285}," [j ",[276,1905,1237],{"class":281},[276,1907,1908],{"class":285}," j ",[276,1910,466],{"class":281},[276,1912,1913],{"class":285}," jobs\n",[276,1915,1916,1918,1920,1923,1925,1928,1930,1932,1935],{"class":105,"line":490},[276,1917,514],{"class":281},[276,1919,517],{"class":281},[276,1921,1922],{"class":285}," (previous.get(j[",[276,1924,501],{"class":355},[276,1926,1927],{"class":285},"], {}).get(",[276,1929,1280],{"class":355},[276,1931,395],{"class":285},[276,1933,1934],{"class":281},"==",[276,1936,1937],{"class":355}," \"ok\"\n",[276,1939,1940,1943,1946,1948,1951,1954,1956,1958],{"class":105,"line":511},[276,1941,1942],{"class":281},"                and",[276,1944,1945],{"class":285}," previous[j[",[276,1947,501],{"class":355},[276,1949,1950],{"class":285},"]].get(",[276,1952,1953],{"class":355},"\"hash\"",[276,1955,395],{"class":285},[276,1957,1934],{"class":281},[276,1959,1960],{"class":285}," job_hash(j)\n",[276,1962,1963,1965,1968,1970],{"class":105,"line":533},[276,1964,1942],{"class":281},[276,1966,1967],{"class":285}," Path(j[",[276,1969,1136],{"class":355},[276,1971,1972],{"class":285},"]).exists())]\n",[276,1974,1975,1977,1979,1982,1985,1988,1990,1992,1995,1998],{"class":105,"line":570},[276,1976,1152],{"class":368},[276,1978,917],{"class":285},[276,1980,1981],{"class":368},"len",[276,1983,1984],{"class":285},"(todo), ",[276,1986,1987],{"class":355},"\"of\"",[276,1989,384],{"class":285},[276,1991,1981],{"class":368},[276,1993,1994],{"class":285},"(jobs), ",[276,1996,1997],{"class":355},"\"jobs need running\"",[276,1999,567],{"class":285},[14,2001,2002,2004,2005,1169],{},[188,2003,190],{}," A hash of the parameters that affect the output detects edited rows: change a distance and the hash changes, so the job reruns even though it succeeded before. Store the hash in the results table (add it to the result rows) so the next run can compare. Checking that the output file still exists catches outputs deleted since. The same idea, applied to long single runs, is covered in ",[21,2006,2008],{"href":2007},"\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Fresume-and-checkpoint-a-batch-run-pyqgis\u002F","resuming and checkpointing a batch run",[154,2010,2012],{"id":2011},"support-a-dry-run","Support a dry run",[14,2014,2015],{},"A dry run validates and prints what would happen without running anything — useful before long batches and for reviewing a colleague's table.",[176,2017,2019],{"className":270,"code":2018,"language":272,"meta":181,"style":181},"import sys\n\nDRY_RUN = \"--dry-run\" in sys.argv\nif DRY_RUN:\n    for job in jobs:\n        state = \"would run\" if job[\"enabled\"].strip() == \"1\" else \"disabled\"\n        print(f\"{job['job_id']:\u003C8} {state:\u003C10} {job['algorithm']} → {job['output']}\")\n    raise SystemExit(0)\n",[183,2020,2021,2028,2032,2048,2057,2067,2096,2157],{"__ignoreMap":181},[276,2022,2023,2025],{"class":105,"line":278},[276,2024,282],{"class":281},[276,2026,2027],{"class":285}," sys\n",[276,2029,2030],{"class":105,"line":289},[276,2031,320],{"emptyLinePlaceholder":319},[276,2033,2034,2037,2040,2043,2045],{"class":105,"line":303},[276,2035,2036],{"class":368},"DRY_RUN",[276,2038,2039],{"class":281}," =",[276,2041,2042],{"class":355}," \"--dry-run\"",[276,2044,597],{"class":281},[276,2046,2047],{"class":285}," sys.argv\n",[276,2049,2050,2052,2055],{"class":105,"line":316},[276,2051,905],{"class":281},[276,2053,2054],{"class":368}," DRY_RUN",[276,2056,656],{"class":285},[276,2058,2059,2061,2063,2065],{"class":105,"line":323},[276,2060,460],{"class":281},[276,2062,1240],{"class":285},[276,2064,466],{"class":281},[276,2066,1245],{"class":285},[276,2068,2069,2072,2074,2077,2080,2082,2084,2086,2088,2090,2093],{"class":105,"line":335},[276,2070,2071],{"class":285},"        state ",[276,2073,329],{"class":281},[276,2075,2076],{"class":355}," \"would run\"",[276,2078,2079],{"class":281}," if",[276,2081,1253],{"class":285},[276,2083,583],{"class":355},[276,2085,1258],{"class":285},[276,2087,1934],{"class":281},[276,2089,1264],{"class":355},[276,2091,2092],{"class":281}," else",[276,2094,2095],{"class":355}," \"disabled\"\n",[276,2097,2098,2101,2103,2105,2107,2109,2111,2113,2115,2117,2119,2121,2124,2127,2129,2131,2133,2135,2137,2139,2142,2144,2146,2149,2151,2153,2155],{"class":105,"line":340},[276,2099,2100],{"class":368},"        print",[276,2102,917],{"class":285},[276,2104,539],{"class":281},[276,2106,743],{"class":355},[276,2108,545],{"class":368},[276,2110,1522],{"class":285},[276,2112,1525],{"class":355},[276,2114,1528],{"class":285},[276,2116,1531],{"class":281},[276,2118,551],{"class":368},[276,2120,1536],{"class":368},[276,2122,2123],{"class":285},"state",[276,2125,2126],{"class":281},":\u003C10",[276,2128,551],{"class":368},[276,2130,1536],{"class":368},[276,2132,1522],{"class":285},[276,2134,682],{"class":355},[276,2136,1528],{"class":285},[276,2138,551],{"class":368},[276,2140,2141],{"class":355}," → ",[276,2143,545],{"class":368},[276,2145,1522],{"class":285},[276,2147,2148],{"class":355},"'output'",[276,2150,1528],{"class":285},[276,2152,551],{"class":368},[276,2154,743],{"class":355},[276,2156,567],{"class":285},[276,2158,2159,2161,2163,2165,2167],{"class":105,"line":362},[276,2160,933],{"class":281},[276,2162,936],{"class":368},[276,2164,917],{"class":285},[276,2166,46],{"class":368},[276,2168,567],{"class":285},[14,2170,2171,2173],{},[188,2172,190],{}," A command-line flag switches between dry and real runs without editing the script. Printing algorithm and output per job shows exactly what a run would touch, including outputs that would overwrite existing files. Running the dry run after every table edit is a cheap habit that catches mistakes such as two jobs writing to the same output.",[154,2175,2177],{"id":2176},"qgis-version-compatibility","QGIS version compatibility",[14,2179,2180,2181,2184],{},"The approach uses only ",[183,2182,2183],{},"processing.run"," and the algorithm registry, which work on QGIS 3.34 LTR, 3.40 LTR and QGIS 4. Parameter names come from each algorithm and are stable across these releases for the native algorithms shown.",[154,2186,2188],{"id":2187},"troubleshooting","Troubleshooting",[159,2190,2191,2199,2205,2211],{},[162,2192,2193,2196,2197,1169],{},[188,2194,2195],{},"The first column name is garbled."," The file has a byte-order mark; read with ",[183,2198,963],{},[162,2200,2201,2204],{},[188,2202,2203],{},"Every row is one big column."," The delimiter is wrong; spreadsheets in many locales use semicolons.",[162,2206,2207,2210],{},[188,2208,2209],{},"Numbers fail to parse."," Decimal commas or units typed into cells; normalise in the converter or tighten the table.",[162,2212,2213,2216],{},[188,2214,2215],{},"Two jobs overwrite each other."," Duplicate outputs; add a uniqueness check on the output column to validation.",[154,2218,2220],{"id":2219},"conclusion","Conclusion",[14,2222,2223],{},"Describe jobs as rows with readable, unit-bearing column names and an enabled flag, validate the whole table before running and report every problem, convert text to typed parameters in one place, run enabled jobs while recording status, counts, timing and errors to a results table, rerun only failed, new or changed jobs, and offer a dry run.",[154,2225,2227],{"id":2226},"frequently-asked-questions","Frequently Asked Questions",[14,2229,2230,2233,2234,2237],{},[188,2231,2232],{},"Can the table live in a spreadsheet instead of CSV?","\nYes — read ",[183,2235,2236],{},".xlsx"," with pandas or openpyxl; CSV is simpler to diff and version.",[14,2239,2240,2243,2244,2247],{},[188,2241,2242],{},"Can a row reference a QGIS project layer instead of a file?","\nYes, by layer name; look it up with ",[183,2245,2246],{},"QgsProject.instance().mapLayersByName"," in the converter.",[14,2249,2250,2253],{},[188,2251,2252],{},"What about Processing's own batch interface?","\nIt loads and saves parameter tables in its own JSON format and suits interactive work; a script suits scheduled and versioned runs.",[14,2255,2256,2259],{},[188,2257,2258],{},"Should the job table be versioned?","\nYes. Keep it in the same repository as the script; a diff of the table shows exactly which jobs changed between runs, which is the audit trail most teams need.",[14,2261,2262,2265],{},[188,2263,2264],{},"How do I run different algorithms per row?","\nKeep an algorithm column and one converter per algorithm, chosen by the row's value.",[154,2267,2269],{"id":2268},"related","Related",[159,2271,2272,2277,2283,2288,2294],{},[162,2273,2274,2276],{},[21,2275,24],{"href":23}," — the guide this recipe belongs to",[162,2278,2279],{},[21,2280,2282],{"href":2281},"\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Fbatch-process-all-layers-in-project-pyqgis\u002F","Batch Process All Layers in a Project with PyQGIS",[162,2284,2285],{},[21,2286,2287],{"href":2007},"Resume and Checkpoint a Batch Run in PyQGIS",[162,2289,2290],{},[21,2291,2293],{"href":2292},"\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Frun-processing-algorithm-from-script\u002F","Run a Processing Algorithm from a Script",[162,2295,2296],{},[21,2297,2299],{"href":2298},"\u002Fspatial-data-processing-automation\u002Fdata-quality-and-topology-validation\u002Fvalidate-attribute-values-against-rules-pyqgis\u002F","Validate Attribute Values Against Rules in PyQGIS",[2301,2302,2303],"style",{},"html pre.shiki code .snl16, html code.shiki .snl16{--shiki-default:#F97583}html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html pre.shiki code .s9osk, html code.shiki .s9osk{--shiki-default:#FFAB70}html pre.shiki code .sjoCn, html code.shiki .sjoCn{--shiki-default:#9AA79F}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":181,"searchDepth":289,"depth":289,"links":2305},[2306,2307,2308,2309,2310,2311,2312,2313,2314,2315,2316,2317],{"id":156,"depth":289,"text":157},{"id":170,"depth":289,"text":171},{"id":206,"depth":289,"text":207},{"id":967,"depth":289,"text":968},{"id":1172,"depth":289,"text":1173},{"id":1693,"depth":289,"text":1694},{"id":2011,"depth":289,"text":2012},{"id":2176,"depth":289,"text":2177},{"id":2187,"depth":289,"text":2188},{"id":2219,"depth":289,"text":2220},{"id":2226,"depth":289,"text":2227},{"id":2268,"depth":289,"text":2269},"Drive Processing runs from a spreadsheet of parameters — one row per job with inputs, values and outputs, validating every row before running, converting text to the types algorithms expect, writing results back into the table, and letting non-programmers maintain the job list.","md",{"slug":2321,"type":2322,"breadcrumb":2323,"datePublished":2324,"dateModified":2324},"run-batch-jobs-from-csv-parameter-table-pyqgis","article","Run Batch Jobs from a CSV Parameter Table","2026-10-02","\u002Fspatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Frun-batch-jobs-from-csv-parameter-table-pyqgis",{"title":5,"description":2318},"spatial-data-processing-automation\u002Fbatch-processing-with-pyqgis\u002Frun-batch-jobs-from-csv-parameter-table-pyqgis\u002Findex","AkJ_V225PVATHsrlvJMTyqPHnroN1i9alHQAgTEgA8s",1790966260765]