[{"data":1,"prerenderedAt":2253},["ShallowReactive",2],{"doc:\u002Fspatial-data-processing-automation\u002Fpyqgis-and-the-python-data-stack\u002Fanalyse-attribute-table-with-pandas-pyqgis":3},{"id":4,"title":5,"body":6,"description":2242,"extension":2243,"meta":2244,"navigation":250,"path":2249,"seo":2250,"stem":2251,"__hash__":2252},"docs\u002Fspatial-data-processing-automation\u002Fpyqgis-and-the-python-data-stack\u002Fanalyse-attribute-table-with-pandas-pyqgis\u002Findex.md","Analyse an Attribute Table with Pandas in PyQGIS",{"type":7,"value":8,"toc":2227},"minimark",[9,13,17,26,173,178,197,201,204,519,529,533,536,725,747,751,754,878,1199,1215,1219,1222,1488,1501,1505,1508,1669,1682,1686,1689,1764,1940,1959,1963,1966,2059,2080,2084,2104,2108,2145,2149,2152,2156,2166,2172,2178,2188,2192,2223],[10,11,5],"h1",{"id":12},"analyse-an-attribute-table-with-pandas-in-pyqgis",[14,15,16],"p",{},"Many questions about spatial data are not spatial at all once the spatial work is done. How many inspections per district by year? Which material has the highest failure rate among pipes older than fifty years? What is the median floor area by building use, compared with last year's survey? QGIS's attribute table and aggregate functions can answer some of these, but pandas answers all of them in a line or two — group-bys, pivots, merges, rolling windows — and then exports the result straight to a spreadsheet.",[14,18,19,20,25],{},"This recipe belongs to ",[21,22,24],"a",{"href":23},"\u002Fspatial-data-processing-automation\u002Fpyqgis-and-the-python-data-stack\u002F","PyQGIS and the Python Data Stack",". It reads an attribute table into a DataFrame without geometry, cleans types, summarises and pivots, joins external tables, writes computed columns back to the layer, and exports results.",[14,27,28],{},[29,30,35,39,43,50,67,76,85,91,97,101,108,116,120,124,128,132,138,142,150,155,159,164,169],"svg",{"viewBox":31,"role":32,"ariaLabel":33,"xmlns":34},"0 0 760 250","img","Attributes read into a DataFrame indexed by feature id, analysed in pandas, exported as summary tables and written back to the layer as new columns","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg",[36,37,38],"title",{},"Attributes out, results back",[40,41,42],"desc",{},"Attributes are read from the layer without geometry into a DataFrame indexed by feature id. Pandas does the analysis: summaries, pivots and joins with spreadsheets. Summary tables go out to Excel or CSV for reports. Computed per-feature columns go back to the layer by feature id in one provider call, where they can be styled and mapped.",[44,45],"rect",{"x":46,"y":46,"width":47,"height":48,"fill":49},"0","760","250","#f6f3ea",[51,52,53],"defs",{},[54,55,62],"marker",{"id":56,"viewBox":57,"refX":58,"refY":59,"markerWidth":60,"markerHeight":60,"orient":61},"pdLoopArrow","0 0 10 10","8","5","7","auto-start-reverse",[63,64],"path",{"d":65,"fill":66},"M0 0 L10 5 L0 10 z","#2f3b35",[68,69,75],"text",{"x":70,"y":71,"style":72,"fill":73,"textAnchor":74},"380","28","text-anchor:middle;font-size:14px;font-family:sans-serif;font-weight:bold","#17211d","middle","Read once, analyse, write back by id",[44,77],{"x":78,"y":79,"width":80,"height":81,"rx":58,"fill":82,"stroke":83,"style":84},"24","96","190","90","#eef7f4","#0f766e","stroke-width:2",[68,86,90],{"x":87,"y":88,"style":89,"fill":83,"textAnchor":74},"119","120.78","text-anchor:middle;font-size:11.5px;font-family:sans-serif;font-weight:bold","layer",[68,92,96],{"x":87,"y":93,"style":94,"fill":95,"textAnchor":74},"144.78","text-anchor:middle;font-size:10.5px;font-family:sans-serif","#59645f","attributes only",[68,98,100],{"x":87,"y":99,"style":94,"fill":95,"textAnchor":74},"168.78","NoGeometry",[102,103],"line",{"x1":104,"y1":105,"x2":106,"y2":105,"stroke":66,"style":107},"214","141","256","stroke-width:1.8;marker-end:url(#pdLoopArrow)",[44,109],{"x":110,"y":111,"width":112,"height":113,"rx":58,"fill":114,"stroke":115,"style":84},"264","72","232","138","#eff3ff","#2563eb",[68,117,119],{"x":70,"y":118,"style":89,"fill":115,"textAnchor":74},"105.78","DataFrame",[68,121,123],{"x":70,"y":122,"style":94,"fill":66,"textAnchor":74},"131.78","index = feature id",[68,125,127],{"x":70,"y":126,"style":94,"fill":66,"textAnchor":74},"157.78","groupby · pivot",[68,129,131],{"x":70,"y":130,"style":94,"fill":66,"textAnchor":74},"183.78","merge · rolling",[102,133],{"x1":134,"y1":135,"x2":136,"y2":137,"stroke":66,"style":107},"496","110","540","80",[102,139],{"x1":134,"y1":140,"x2":136,"y2":141,"stroke":66,"style":107},"172","202",[44,143],{"x":144,"y":145,"width":146,"height":147,"rx":58,"fill":148,"stroke":149,"style":84},"548","48","188","64","#fdf2e2","#b45309",[68,151,154],{"x":152,"y":153,"style":89,"fill":149,"textAnchor":74},"642","73.805","summary tables",[68,156,158],{"x":152,"y":157,"style":94,"fill":95,"textAnchor":74},"93.755","Excel · CSV",[44,160],{"x":144,"y":161,"width":146,"height":147,"rx":58,"fill":162,"stroke":163,"style":84},"170","#e8efe6","#15803d",[68,165,168],{"x":152,"y":166,"style":89,"fill":167,"textAnchor":74},"195.805","#166534","new columns",[68,170,172],{"x":152,"y":171,"style":94,"fill":95,"textAnchor":74},"215.755","back by feature id",[174,175,177],"h2",{"id":176},"prerequisites","Prerequisites",[179,180,181,190],"ul",{},[182,183,184,185,189],"li",{},"QGIS 3.34 LTR or newer, or the QGIS 4 series. Pandas ships with most QGIS installers; check with ",[186,187,188],"code",{},"import pandas; pandas.__version__"," in the Python console.",[182,191,192,193,196],{},"For Excel output, ",[186,194,195],{},"openpyxl"," installed into QGIS's Python.",[174,198,200],{"id":199},"read-attributes-into-a-dataframe","Read attributes into a DataFrame",[14,202,203],{},"Geometry is often the largest part of a feature. Skipping it, and indexing rows by feature id, gives a lean frame that can be joined back to the layer later.",[205,206,211],"pre",{"className":207,"code":208,"language":209,"meta":210,"style":210},"language-python shiki shiki-themes github-dark","import pandas as pd\nfrom qgis.core import QgsProject, QgsFeatureRequest\n\npipes = QgsProject.instance().mapLayersByName(\"water_pipes\")[0]\n\ndef attributes_frame(layer, fields=None, expression=None):\n    names = fields or layer.fields().names()\n    req = QgsFeatureRequest().setFlags(QgsFeatureRequest.NoGeometry)\n    req.setSubsetOfAttributes(names, layer.fields())\n    if expression:\n        req.setFilterExpression(expression)\n    ids, rows = [], []\n    for f in layer.getFeatures(req):\n        ids.append(f.id())\n        rows.append([f[n] for n in names])\n    return pd.DataFrame(rows, columns=names, index=pd.Index(ids, name=\"fid\"))\n\ndf = attributes_frame(pipes, [\"asset_id\", \"material\", \"diameter_mm\",\n                              \"install_year\", \"district\", \"bursts_5y\", \"length_m\"])\nprint(df.shape)\nprint(df.head())\n","python","",[186,212,213,231,245,252,277,282,310,327,338,344,353,359,370,385,391,408,445,450,478,502,511],{"__ignoreMap":210},[214,215,217,221,225,228],"span",{"class":102,"line":216},1,[214,218,220],{"class":219},"snl16","import",[214,222,224],{"class":223},"s95oV"," pandas ",[214,226,227],{"class":219},"as",[214,229,230],{"class":223}," pd\n",[214,232,234,237,240,242],{"class":102,"line":233},2,[214,235,236],{"class":219},"from",[214,238,239],{"class":223}," qgis.core ",[214,241,220],{"class":219},[214,243,244],{"class":223}," QgsProject, QgsFeatureRequest\n",[214,246,248],{"class":102,"line":247},3,[214,249,251],{"emptyLinePlaceholder":250},true,"\n",[214,253,255,258,261,264,268,271,274],{"class":102,"line":254},4,[214,256,257],{"class":223},"pipes ",[214,259,260],{"class":219},"=",[214,262,263],{"class":223}," QgsProject.instance().mapLayersByName(",[214,265,267],{"class":266},"sU2Wk","\"water_pipes\"",[214,269,270],{"class":223},")[",[214,272,46],{"class":273},"sDLfK",[214,275,276],{"class":223},"]\n",[214,278,280],{"class":102,"line":279},5,[214,281,251],{"emptyLinePlaceholder":250},[214,283,285,288,292,295,297,300,303,305,307],{"class":102,"line":284},6,[214,286,287],{"class":219},"def",[214,289,291],{"class":290},"svObZ"," attributes_frame",[214,293,294],{"class":223},"(layer, fields",[214,296,260],{"class":219},[214,298,299],{"class":273},"None",[214,301,302],{"class":223},", expression",[214,304,260],{"class":219},[214,306,299],{"class":273},[214,308,309],{"class":223},"):\n",[214,311,313,316,318,321,324],{"class":102,"line":312},7,[214,314,315],{"class":223},"    names ",[214,317,260],{"class":219},[214,319,320],{"class":223}," fields ",[214,322,323],{"class":219},"or",[214,325,326],{"class":223}," layer.fields().names()\n",[214,328,330,333,335],{"class":102,"line":329},8,[214,331,332],{"class":223},"    req ",[214,334,260],{"class":219},[214,336,337],{"class":223}," QgsFeatureRequest().setFlags(QgsFeatureRequest.NoGeometry)\n",[214,339,341],{"class":102,"line":340},9,[214,342,343],{"class":223},"    req.setSubsetOfAttributes(names, layer.fields())\n",[214,345,347,350],{"class":102,"line":346},10,[214,348,349],{"class":219},"    if",[214,351,352],{"class":223}," expression:\n",[214,354,356],{"class":102,"line":355},11,[214,357,358],{"class":223},"        req.setFilterExpression(expression)\n",[214,360,362,365,367],{"class":102,"line":361},12,[214,363,364],{"class":223},"    ids, rows ",[214,366,260],{"class":219},[214,368,369],{"class":223}," [], []\n",[214,371,373,376,379,382],{"class":102,"line":372},13,[214,374,375],{"class":219},"    for",[214,377,378],{"class":223}," f ",[214,380,381],{"class":219},"in",[214,383,384],{"class":223}," layer.getFeatures(req):\n",[214,386,388],{"class":102,"line":387},14,[214,389,390],{"class":223},"        ids.append(f.id())\n",[214,392,394,397,400,403,405],{"class":102,"line":393},15,[214,395,396],{"class":223},"        rows.append([f[n] ",[214,398,399],{"class":219},"for",[214,401,402],{"class":223}," n ",[214,404,381],{"class":219},[214,406,407],{"class":223}," names])\n",[214,409,411,414,417,421,423,426,429,431,434,437,439,442],{"class":102,"line":410},16,[214,412,413],{"class":219},"    return",[214,415,416],{"class":223}," pd.DataFrame(rows, ",[214,418,420],{"class":419},"s9osk","columns",[214,422,260],{"class":219},[214,424,425],{"class":223},"names, ",[214,427,428],{"class":419},"index",[214,430,260],{"class":219},[214,432,433],{"class":223},"pd.Index(ids, ",[214,435,436],{"class":419},"name",[214,438,260],{"class":219},[214,440,441],{"class":266},"\"fid\"",[214,443,444],{"class":223},"))\n",[214,446,448],{"class":102,"line":447},17,[214,449,251],{"emptyLinePlaceholder":250},[214,451,453,456,458,461,464,467,470,472,475],{"class":102,"line":452},18,[214,454,455],{"class":223},"df ",[214,457,260],{"class":219},[214,459,460],{"class":223}," attributes_frame(pipes, [",[214,462,463],{"class":266},"\"asset_id\"",[214,465,466],{"class":223},", ",[214,468,469],{"class":266},"\"material\"",[214,471,466],{"class":223},[214,473,474],{"class":266},"\"diameter_mm\"",[214,476,477],{"class":223},",\n",[214,479,481,484,486,489,491,494,496,499],{"class":102,"line":480},19,[214,482,483],{"class":266},"                              \"install_year\"",[214,485,466],{"class":223},[214,487,488],{"class":266},"\"district\"",[214,490,466],{"class":223},[214,492,493],{"class":266},"\"bursts_5y\"",[214,495,466],{"class":223},[214,497,498],{"class":266},"\"length_m\"",[214,500,501],{"class":223},"])\n",[214,503,505,508],{"class":102,"line":504},20,[214,506,507],{"class":273},"print",[214,509,510],{"class":223},"(df.shape)\n",[214,512,514,516],{"class":102,"line":513},21,[214,515,507],{"class":273},[214,517,518],{"class":223},"(df.head())\n",[14,520,521,525,526,528],{},[522,523,524],"strong",{},"Breakdown:"," ",[186,527,100],{}," and an attribute subset push the work down to the provider, so on a database only the needed columns cross the network. Indexing by feature id rather than a 0-based range is the key design choice: it makes writing results back trivial and survives filtering and sorting. Reading values by name keeps the code independent of field order.",[174,530,532],{"id":531},"clean-types-before-analysing","Clean types before analysing",[14,534,535],{},"QGIS hands pandas NULL QVariants and Qt date objects on QGIS 3. Cleaning them once, right after reading, avoids object-dtype columns that silently break numeric operations.",[205,537,539],{"className":207,"code":538,"language":209,"meta":210,"style":210},"def clean(df):\n    def py(v):\n        if v is None or (hasattr(v, \"isNull\") and v.isNull()):\n            return None\n        if hasattr(v, \"toPyDateTime\"):\n            return v.toPyDateTime()\n        if hasattr(v, \"toPyDate\"):\n            return v.toPyDate()\n        return v\n    out = df.apply(lambda col: col.map(py) if col.dtype == object else col)\n    return out.convert_dtypes()\n\ndf = clean(df)\nprint(df.dtypes)\nprint(df.isna().sum())\n",[186,540,541,551,562,600,608,622,629,642,649,657,691,698,702,711,718],{"__ignoreMap":210},[214,542,543,545,548],{"class":102,"line":216},[214,544,287],{"class":219},[214,546,547],{"class":290}," clean",[214,549,550],{"class":223},"(df):\n",[214,552,553,556,559],{"class":102,"line":233},[214,554,555],{"class":219},"    def",[214,557,558],{"class":290}," py",[214,560,561],{"class":223},"(v):\n",[214,563,564,567,570,573,576,579,582,585,588,591,594,597],{"class":102,"line":247},[214,565,566],{"class":219},"        if",[214,568,569],{"class":223}," v ",[214,571,572],{"class":219},"is",[214,574,575],{"class":273}," None",[214,577,578],{"class":219}," or",[214,580,581],{"class":223}," (",[214,583,584],{"class":273},"hasattr",[214,586,587],{"class":223},"(v, ",[214,589,590],{"class":266},"\"isNull\"",[214,592,593],{"class":223},") ",[214,595,596],{"class":219},"and",[214,598,599],{"class":223}," v.isNull()):\n",[214,601,602,605],{"class":102,"line":254},[214,603,604],{"class":219},"            return",[214,606,607],{"class":273}," None\n",[214,609,610,612,615,617,620],{"class":102,"line":279},[214,611,566],{"class":219},[214,613,614],{"class":273}," hasattr",[214,616,587],{"class":223},[214,618,619],{"class":266},"\"toPyDateTime\"",[214,621,309],{"class":223},[214,623,624,626],{"class":102,"line":284},[214,625,604],{"class":219},[214,627,628],{"class":223}," v.toPyDateTime()\n",[214,630,631,633,635,637,640],{"class":102,"line":312},[214,632,566],{"class":219},[214,634,614],{"class":273},[214,636,587],{"class":223},[214,638,639],{"class":266},"\"toPyDate\"",[214,641,309],{"class":223},[214,643,644,646],{"class":102,"line":329},[214,645,604],{"class":219},[214,647,648],{"class":223}," v.toPyDate()\n",[214,650,651,654],{"class":102,"line":340},[214,652,653],{"class":219},"        return",[214,655,656],{"class":223}," v\n",[214,658,659,662,664,667,670,673,676,679,682,685,688],{"class":102,"line":346},[214,660,661],{"class":223},"    out ",[214,663,260],{"class":219},[214,665,666],{"class":223}," df.apply(",[214,668,669],{"class":219},"lambda",[214,671,672],{"class":223}," col: col.map(py) ",[214,674,675],{"class":219},"if",[214,677,678],{"class":223}," col.dtype ",[214,680,681],{"class":219},"==",[214,683,684],{"class":273}," object",[214,686,687],{"class":219}," else",[214,689,690],{"class":223}," col)\n",[214,692,693,695],{"class":102,"line":355},[214,694,413],{"class":219},[214,696,697],{"class":223}," out.convert_dtypes()\n",[214,699,700],{"class":102,"line":361},[214,701,251],{"emptyLinePlaceholder":250},[214,703,704,706,708],{"class":102,"line":372},[214,705,455],{"class":223},[214,707,260],{"class":219},[214,709,710],{"class":223}," clean(df)\n",[214,712,713,715],{"class":102,"line":387},[214,714,507],{"class":273},[214,716,717],{"class":223},"(df.dtypes)\n",[214,719,720,722],{"class":102,"line":393},[214,721,507],{"class":273},[214,723,724],{"class":223},"(df.isna().sum())\n",[14,726,727,729,730,733,734,737,738,741,742,746],{},[522,728,524],{}," Mapping only object columns leaves numeric columns fast. ",[186,731,732],{},"convert_dtypes"," chooses pandas' nullable types, so an integer column with missing values stays ",[186,735,736],{},"Int64"," instead of turning into floats, and text becomes ",[186,739,740],{},"string",". The missing-value count per column is worth printing every time: it tells you which analyses will quietly drop rows. The underlying NULL rules are in ",[21,743,745],{"href":744},"\u002Fpyqgis-fundamentals-environment-setup\u002Ffeatures-geometries-and-memory-layers\u002Fhandle-null-values-and-qvariant-pyqgis\u002F","handling NULL values and QVariant",".",[174,748,750],{"id":749},"summarise-with-group-bys-and-pivots","Summarise with group-bys and pivots",[14,752,753],{},"Group-bys answer \"per category\" questions; pivot tables lay two categories against each other. Both are one line each.",[14,755,756],{},[29,757,759,762,765,767,770,777,781,784,788,795,799,806,809,812,815,819,821,824,827,830,832,835,838,842,844,846,849,853,860,866,870,874],{"viewBox":31,"role":32,"ariaLabel":758,"xmlns":34},"A pivot of burst rates by pipe material and installation decade, highlighting old cast iron as the worst combination",[36,760,761],{},"A pivot of burst rate by material and age band",[40,763,764],{},"A pivot table with materials as rows, cast iron, PVC, polyethylene and ductile iron, and installation decade bands as columns. Cells hold bursts per 100 kilometres per year. Cast iron from before 1960 has the highest rate and is highlighted; modern polyethylene has the lowest. The table directs renewal planning to the worst combination.",[44,766],{"x":46,"y":46,"width":47,"height":48,"fill":49},[68,768,769],{"x":70,"y":71,"style":72,"fill":73,"textAnchor":74},"Bursts per 100 km per year",[68,771,776],{"x":772,"y":773,"style":774,"fill":73,"textAnchor":775},"150","76","text-anchor:end;font-size:10.5px;font-family:sans-serif;font-weight:bold","end","material",[68,778,780],{"x":48,"y":773,"style":779,"fill":73,"textAnchor":74},"text-anchor:middle;font-size:10.5px;font-family:sans-serif;font-weight:bold","\u003C 1960",[68,782,783],{"x":70,"y":773,"style":779,"fill":73,"textAnchor":74},"1960–89",[68,785,787],{"x":786,"y":773,"style":779,"fill":73,"textAnchor":74},"510","1990–",[102,789],{"x1":790,"y1":791,"x2":792,"y2":791,"stroke":793,"style":794},"60","86","600","#d9d3c4","stroke-width:1.5",[68,796,798],{"x":772,"y":135,"style":797,"fill":66,"textAnchor":775},"text-anchor:end;font-size:10.5px;font-family:sans-serif","cast iron",[44,800],{"x":801,"y":802,"width":81,"height":803,"rx":804,"fill":148,"stroke":805,"style":794},"205","95","22","3","#b91c1c",[68,807,808],{"x":48,"y":135,"style":779,"fill":805,"textAnchor":74},"41.2",[68,810,811],{"x":70,"y":135,"style":94,"fill":66,"textAnchor":74},"18.5",[68,813,814],{"x":786,"y":135,"style":94,"fill":66,"textAnchor":74},"—",[68,816,818],{"x":772,"y":817,"style":797,"fill":66,"textAnchor":775},"140","ductile iron",[68,820,814],{"x":48,"y":817,"style":94,"fill":66,"textAnchor":74},[68,822,823],{"x":70,"y":817,"style":94,"fill":66,"textAnchor":74},"9.7",[68,825,826],{"x":786,"y":817,"style":94,"fill":66,"textAnchor":74},"4.1",[68,828,829],{"x":772,"y":161,"style":797,"fill":66,"textAnchor":775},"PVC",[68,831,814],{"x":48,"y":161,"style":94,"fill":66,"textAnchor":74},[68,833,834],{"x":70,"y":161,"style":94,"fill":66,"textAnchor":74},"12.3",[68,836,837],{"x":786,"y":161,"style":94,"fill":66,"textAnchor":74},"6.0",[68,839,841],{"x":772,"y":840,"style":797,"fill":66,"textAnchor":775},"200","polyethylene",[68,843,814],{"x":48,"y":840,"style":94,"fill":66,"textAnchor":74},[68,845,814],{"x":70,"y":840,"style":94,"fill":66,"textAnchor":74},[68,847,848],{"x":786,"y":840,"style":94,"fill":167,"textAnchor":74},"1.8",[68,850,852],{"x":70,"y":112,"style":851,"fill":95,"textAnchor":74},"text-anchor:middle;font-size:10.0px;font-family:sans-serif","illustrative values",[44,854],{"x":855,"y":856,"width":857,"height":858,"rx":58,"fill":859,"stroke":95,"style":84},"620","92","116","118","#fffdf7",[68,861,865],{"x":862,"y":863,"style":864,"fill":73,"textAnchor":74},"678","121.6","text-anchor:middle;font-size:11.0px;font-family:sans-serif;font-weight:bold","renew",[68,867,869],{"x":862,"y":868,"style":851,"fill":66,"textAnchor":74},"143.6","first:",[68,871,873],{"x":862,"y":872,"style":851,"fill":66,"textAnchor":74},"165.6","old cast",[68,875,877],{"x":862,"y":876,"style":851,"fill":66,"textAnchor":74},"187.6","iron",[205,879,881],{"className":207,"code":880,"language":209,"meta":210,"style":210},"df[\"decade_band\"] = pd.cut(df[\"install_year\"], bins=[0, 1959, 1989, 2100],\n                           labels=[\"\u003C 1960\", \"1960–89\", \"1990–\"])\nby_material = (df.groupby(\"material\")\n                 .agg(km=(\"length_m\", lambda s: s.sum() \u002F 1000),\n                      bursts=(\"bursts_5y\", \"sum\"))\n                 .assign(rate=lambda t: t.bursts \u002F t.km \u002F 5 * 100)\n                 .sort_values(\"rate\", ascending=False))\nprint(by_material.round(1))\n\npivot = df.pivot_table(index=\"material\", columns=\"decade_band\",\n                       values=[\"bursts_5y\", \"length_m\"], aggfunc=\"sum\", observed=False)\nrate = pivot[\"bursts_5y\"] \u002F (pivot[\"length_m\"] \u002F 1000) \u002F 5 * 100\nprint(rate.round(1))\n",[186,882,883,933,957,972,1003,1021,1053,1073,1085,1089,1115,1150,1188],{"__ignoreMap":210},[214,884,885,888,891,894,896,899,902,905,908,910,913,915,917,920,922,925,927,930],{"class":102,"line":216},[214,886,887],{"class":223},"df[",[214,889,890],{"class":266},"\"decade_band\"",[214,892,893],{"class":223},"] ",[214,895,260],{"class":219},[214,897,898],{"class":223}," pd.cut(df[",[214,900,901],{"class":266},"\"install_year\"",[214,903,904],{"class":223},"], ",[214,906,907],{"class":419},"bins",[214,909,260],{"class":219},[214,911,912],{"class":223},"[",[214,914,46],{"class":273},[214,916,466],{"class":223},[214,918,919],{"class":273},"1959",[214,921,466],{"class":223},[214,923,924],{"class":273},"1989",[214,926,466],{"class":223},[214,928,929],{"class":273},"2100",[214,931,932],{"class":223},"],\n",[214,934,935,938,940,942,945,947,950,952,955],{"class":102,"line":233},[214,936,937],{"class":419},"                           labels",[214,939,260],{"class":219},[214,941,912],{"class":223},[214,943,944],{"class":266},"\"\u003C 1960\"",[214,946,466],{"class":223},[214,948,949],{"class":266},"\"1960–89\"",[214,951,466],{"class":223},[214,953,954],{"class":266},"\"1990–\"",[214,956,501],{"class":223},[214,958,959,962,964,967,969],{"class":102,"line":247},[214,960,961],{"class":223},"by_material ",[214,963,260],{"class":219},[214,965,966],{"class":223}," (df.groupby(",[214,968,469],{"class":266},[214,970,971],{"class":223},")\n",[214,973,974,977,980,982,985,987,989,991,994,997,1000],{"class":102,"line":254},[214,975,976],{"class":223},"                 .agg(",[214,978,979],{"class":419},"km",[214,981,260],{"class":219},[214,983,984],{"class":223},"(",[214,986,498],{"class":266},[214,988,466],{"class":223},[214,990,669],{"class":219},[214,992,993],{"class":223}," s: s.sum() ",[214,995,996],{"class":219},"\u002F",[214,998,999],{"class":273}," 1000",[214,1001,1002],{"class":223},"),\n",[214,1004,1005,1008,1010,1012,1014,1016,1019],{"class":102,"line":279},[214,1006,1007],{"class":419},"                      bursts",[214,1009,260],{"class":219},[214,1011,984],{"class":223},[214,1013,493],{"class":266},[214,1015,466],{"class":223},[214,1017,1018],{"class":266},"\"sum\"",[214,1020,444],{"class":223},[214,1022,1023,1026,1029,1032,1035,1037,1040,1042,1045,1048,1051],{"class":102,"line":284},[214,1024,1025],{"class":223},"                 .assign(",[214,1027,1028],{"class":419},"rate",[214,1030,1031],{"class":219},"=lambda",[214,1033,1034],{"class":223}," t: t.bursts ",[214,1036,996],{"class":219},[214,1038,1039],{"class":223}," t.km ",[214,1041,996],{"class":219},[214,1043,1044],{"class":273}," 5",[214,1046,1047],{"class":219}," *",[214,1049,1050],{"class":273}," 100",[214,1052,971],{"class":223},[214,1054,1055,1058,1061,1063,1066,1068,1071],{"class":102,"line":312},[214,1056,1057],{"class":223},"                 .sort_values(",[214,1059,1060],{"class":266},"\"rate\"",[214,1062,466],{"class":223},[214,1064,1065],{"class":419},"ascending",[214,1067,260],{"class":219},[214,1069,1070],{"class":273},"False",[214,1072,444],{"class":223},[214,1074,1075,1077,1080,1083],{"class":102,"line":329},[214,1076,507],{"class":273},[214,1078,1079],{"class":223},"(by_material.round(",[214,1081,1082],{"class":273},"1",[214,1084,444],{"class":223},[214,1086,1087],{"class":102,"line":340},[214,1088,251],{"emptyLinePlaceholder":250},[214,1090,1091,1094,1096,1099,1101,1103,1105,1107,1109,1111,1113],{"class":102,"line":346},[214,1092,1093],{"class":223},"pivot ",[214,1095,260],{"class":219},[214,1097,1098],{"class":223}," df.pivot_table(",[214,1100,428],{"class":419},[214,1102,260],{"class":219},[214,1104,469],{"class":266},[214,1106,466],{"class":223},[214,1108,420],{"class":419},[214,1110,260],{"class":219},[214,1112,890],{"class":266},[214,1114,477],{"class":223},[214,1116,1117,1120,1122,1124,1126,1128,1130,1132,1135,1137,1139,1141,1144,1146,1148],{"class":102,"line":355},[214,1118,1119],{"class":419},"                       values",[214,1121,260],{"class":219},[214,1123,912],{"class":223},[214,1125,493],{"class":266},[214,1127,466],{"class":223},[214,1129,498],{"class":266},[214,1131,904],{"class":223},[214,1133,1134],{"class":419},"aggfunc",[214,1136,260],{"class":219},[214,1138,1018],{"class":266},[214,1140,466],{"class":223},[214,1142,1143],{"class":419},"observed",[214,1145,260],{"class":219},[214,1147,1070],{"class":273},[214,1149,971],{"class":223},[214,1151,1152,1155,1157,1160,1162,1164,1166,1169,1171,1173,1175,1177,1179,1181,1183,1185],{"class":102,"line":361},[214,1153,1154],{"class":223},"rate ",[214,1156,260],{"class":219},[214,1158,1159],{"class":223}," pivot[",[214,1161,493],{"class":266},[214,1163,893],{"class":223},[214,1165,996],{"class":219},[214,1167,1168],{"class":223}," (pivot[",[214,1170,498],{"class":266},[214,1172,893],{"class":223},[214,1174,996],{"class":219},[214,1176,999],{"class":273},[214,1178,593],{"class":223},[214,1180,996],{"class":219},[214,1182,1044],{"class":273},[214,1184,1047],{"class":219},[214,1186,1187],{"class":273}," 100\n",[214,1189,1190,1192,1195,1197],{"class":102,"line":372},[214,1191,507],{"class":273},[214,1193,1194],{"class":223},"(rate.round(",[214,1196,1082],{"class":273},[214,1198,444],{"class":223},[14,1200,1201,525,1203,1206,1207,1210,1211,1214],{},[522,1202,524],{},[186,1204,1205],{},"pd.cut"," turns installation years into bands, the natural unit for an age analysis. Named aggregation (",[186,1208,1209],{},"km=(...)",") keeps output columns readable, and ",[186,1212,1213],{},"assign"," computes a rate from the aggregated totals — the correct way, since averaging per-pipe rates would weight a 2 m pipe like a 2 km one. The pivot sums bursts and lengths separately per material and band, then divides, again giving length-weighted rates. These are the tables a renewal plan is built from.",[174,1216,1218],{"id":1217},"compare-periods-and-spot-changes","Compare periods and spot changes",[14,1220,1221],{},"Attribute tables with dates — inspections, readings, incidents — invite questions about change: which districts got worse this year, which assets have not been inspected since the last cycle. Pandas time handling makes these comparisons short and explicit.",[205,1223,1225],{"className":207,"code":1224,"language":209,"meta":210,"style":210},"insp = clean(attributes_frame(\n    QgsProject.instance().mapLayersByName(\"inspections\")[0],\n    [\"asset_id\", \"district\", \"inspected_on\", \"condition_score\"]))\ninsp[\"inspected_on\"] = pd.to_datetime(insp[\"inspected_on\"])\ninsp[\"year\"] = insp[\"inspected_on\"].dt.year\n\nyearly = insp.pivot_table(index=\"district\", columns=\"year\",\n                          values=\"condition_score\", aggfunc=\"mean\")\nchange = (yearly[2026] - yearly[2025]).sort_values()\nprint(\"largest deterioration:\\n\", change.head(5).round(2))\n\nlatest = insp.sort_values(\"inspected_on\").groupby(\"asset_id\").tail(1)\noverdue = latest[latest[\"inspected_on\"] \u003C pd.Timestamp(\"2024-10-01\")]\nprint(len(overdue), \"assets not inspected in the last two years\")\n",[186,1226,1227,1237,1251,1275,1293,1312,1316,1342,1362,1389,1417,1421,1445,1471],{"__ignoreMap":210},[214,1228,1229,1232,1234],{"class":102,"line":216},[214,1230,1231],{"class":223},"insp ",[214,1233,260],{"class":219},[214,1235,1236],{"class":223}," clean(attributes_frame(\n",[214,1238,1239,1242,1245,1247,1249],{"class":102,"line":233},[214,1240,1241],{"class":223},"    QgsProject.instance().mapLayersByName(",[214,1243,1244],{"class":266},"\"inspections\"",[214,1246,270],{"class":223},[214,1248,46],{"class":273},[214,1250,932],{"class":223},[214,1252,1253,1256,1258,1260,1262,1264,1267,1269,1272],{"class":102,"line":247},[214,1254,1255],{"class":223},"    [",[214,1257,463],{"class":266},[214,1259,466],{"class":223},[214,1261,488],{"class":266},[214,1263,466],{"class":223},[214,1265,1266],{"class":266},"\"inspected_on\"",[214,1268,466],{"class":223},[214,1270,1271],{"class":266},"\"condition_score\"",[214,1273,1274],{"class":223},"]))\n",[214,1276,1277,1280,1282,1284,1286,1289,1291],{"class":102,"line":254},[214,1278,1279],{"class":223},"insp[",[214,1281,1266],{"class":266},[214,1283,893],{"class":223},[214,1285,260],{"class":219},[214,1287,1288],{"class":223}," pd.to_datetime(insp[",[214,1290,1266],{"class":266},[214,1292,501],{"class":223},[214,1294,1295,1297,1300,1302,1304,1307,1309],{"class":102,"line":279},[214,1296,1279],{"class":223},[214,1298,1299],{"class":266},"\"year\"",[214,1301,893],{"class":223},[214,1303,260],{"class":219},[214,1305,1306],{"class":223}," insp[",[214,1308,1266],{"class":266},[214,1310,1311],{"class":223},"].dt.year\n",[214,1313,1314],{"class":102,"line":284},[214,1315,251],{"emptyLinePlaceholder":250},[214,1317,1318,1321,1323,1326,1328,1330,1332,1334,1336,1338,1340],{"class":102,"line":312},[214,1319,1320],{"class":223},"yearly ",[214,1322,260],{"class":219},[214,1324,1325],{"class":223}," insp.pivot_table(",[214,1327,428],{"class":419},[214,1329,260],{"class":219},[214,1331,488],{"class":266},[214,1333,466],{"class":223},[214,1335,420],{"class":419},[214,1337,260],{"class":219},[214,1339,1299],{"class":266},[214,1341,477],{"class":223},[214,1343,1344,1347,1349,1351,1353,1355,1357,1360],{"class":102,"line":329},[214,1345,1346],{"class":419},"                          values",[214,1348,260],{"class":219},[214,1350,1271],{"class":266},[214,1352,466],{"class":223},[214,1354,1134],{"class":419},[214,1356,260],{"class":219},[214,1358,1359],{"class":266},"\"mean\"",[214,1361,971],{"class":223},[214,1363,1364,1367,1369,1372,1375,1377,1380,1383,1386],{"class":102,"line":340},[214,1365,1366],{"class":223},"change ",[214,1368,260],{"class":219},[214,1370,1371],{"class":223}," (yearly[",[214,1373,1374],{"class":273},"2026",[214,1376,893],{"class":223},[214,1378,1379],{"class":219},"-",[214,1381,1382],{"class":223}," yearly[",[214,1384,1385],{"class":273},"2025",[214,1387,1388],{"class":223},"]).sort_values()\n",[214,1390,1391,1393,1395,1398,1401,1404,1407,1409,1412,1415],{"class":102,"line":346},[214,1392,507],{"class":273},[214,1394,984],{"class":223},[214,1396,1397],{"class":266},"\"largest deterioration:",[214,1399,1400],{"class":273},"\\n",[214,1402,1403],{"class":266},"\"",[214,1405,1406],{"class":223},", change.head(",[214,1408,59],{"class":273},[214,1410,1411],{"class":223},").round(",[214,1413,1414],{"class":273},"2",[214,1416,444],{"class":223},[214,1418,1419],{"class":102,"line":355},[214,1420,251],{"emptyLinePlaceholder":250},[214,1422,1423,1426,1428,1431,1433,1436,1438,1441,1443],{"class":102,"line":361},[214,1424,1425],{"class":223},"latest ",[214,1427,260],{"class":219},[214,1429,1430],{"class":223}," insp.sort_values(",[214,1432,1266],{"class":266},[214,1434,1435],{"class":223},").groupby(",[214,1437,463],{"class":266},[214,1439,1440],{"class":223},").tail(",[214,1442,1082],{"class":273},[214,1444,971],{"class":223},[214,1446,1447,1450,1452,1455,1457,1459,1462,1465,1468],{"class":102,"line":372},[214,1448,1449],{"class":223},"overdue ",[214,1451,260],{"class":219},[214,1453,1454],{"class":223}," latest[latest[",[214,1456,1266],{"class":266},[214,1458,893],{"class":223},[214,1460,1461],{"class":219},"\u003C",[214,1463,1464],{"class":223}," pd.Timestamp(",[214,1466,1467],{"class":266},"\"2024-10-01\"",[214,1469,1470],{"class":223},")]\n",[214,1472,1473,1475,1477,1480,1483,1486],{"class":102,"line":387},[214,1474,507],{"class":273},[214,1476,984],{"class":223},[214,1478,1479],{"class":273},"len",[214,1481,1482],{"class":223},"(overdue), ",[214,1484,1485],{"class":266},"\"assets not inspected in the last two years\"",[214,1487,971],{"class":223},[14,1489,1490,1492,1493,1496,1497,1500],{},[522,1491,524],{}," Converting the date column with ",[186,1494,1495],{},"pd.to_datetime"," unlocks the ",[186,1498,1499],{},".dt"," accessor for years, months and weekdays. A pivot of mean scores by district and year, followed by a column difference, ranks districts by change in one expression. Sorting by date and taking the last row per asset gives each asset's most recent inspection — the basis for an overdue list, which can then be written back to the layer as a flag and mapped. Comparing means is only fair when the same kinds of assets were inspected in both years; if inspection effort shifted between districts, compare like with like first.",[174,1502,1504],{"id":1503},"join-external-tables","Join external tables",[14,1506,1507],{},"Attribute analysis often needs data that is not in the layer: a cost table per material, last year's survey, a lookup of district names. Pandas merges handle it, keeping the feature-id index intact.",[205,1509,1511],{"className":207,"code":1510,"language":209,"meta":210,"style":210},"costs = pd.read_excel(\"\u002Fdata\u002Freference\u002Frenewal_costs.xlsx\")    # material, eur_per_m\nmerged = df.reset_index().merge(costs, on=\"material\", how=\"left\").set_index(\"fid\")\nmissing = merged[\"eur_per_m\"].isna().sum()\nif missing:\n    print(f\"{missing} pipes have no cost for their material:\",\n          merged.loc[merged.eur_per_m.isna(), \"material\"].unique())\nmerged[\"renewal_eur\"] = merged[\"length_m\"] * merged[\"eur_per_m\"]\nprint(merged.groupby(\"district\")[\"renewal_eur\"].sum().sort_values().tail())\n",[186,1512,1513,1533,1567,1583,1590,1616,1626,1653],{"__ignoreMap":210},[214,1514,1515,1518,1520,1523,1526,1529],{"class":102,"line":216},[214,1516,1517],{"class":223},"costs ",[214,1519,260],{"class":219},[214,1521,1522],{"class":223}," pd.read_excel(",[214,1524,1525],{"class":266},"\"\u002Fdata\u002Freference\u002Frenewal_costs.xlsx\"",[214,1527,1528],{"class":223},")    ",[214,1530,1532],{"class":1531},"sjoCn","# material, eur_per_m\n",[214,1534,1535,1538,1540,1543,1546,1548,1550,1552,1555,1557,1560,1563,1565],{"class":102,"line":233},[214,1536,1537],{"class":223},"merged ",[214,1539,260],{"class":219},[214,1541,1542],{"class":223}," df.reset_index().merge(costs, ",[214,1544,1545],{"class":419},"on",[214,1547,260],{"class":219},[214,1549,469],{"class":266},[214,1551,466],{"class":223},[214,1553,1554],{"class":419},"how",[214,1556,260],{"class":219},[214,1558,1559],{"class":266},"\"left\"",[214,1561,1562],{"class":223},").set_index(",[214,1564,441],{"class":266},[214,1566,971],{"class":223},[214,1568,1569,1572,1574,1577,1580],{"class":102,"line":247},[214,1570,1571],{"class":223},"missing ",[214,1573,260],{"class":219},[214,1575,1576],{"class":223}," merged[",[214,1578,1579],{"class":266},"\"eur_per_m\"",[214,1581,1582],{"class":223},"].isna().sum()\n",[214,1584,1585,1587],{"class":102,"line":254},[214,1586,675],{"class":219},[214,1588,1589],{"class":223}," missing:\n",[214,1591,1592,1595,1597,1600,1602,1605,1608,1611,1614],{"class":102,"line":279},[214,1593,1594],{"class":273},"    print",[214,1596,984],{"class":223},[214,1598,1599],{"class":219},"f",[214,1601,1403],{"class":266},[214,1603,1604],{"class":273},"{",[214,1606,1607],{"class":223},"missing",[214,1609,1610],{"class":273},"}",[214,1612,1613],{"class":266}," pipes have no cost for their material:\"",[214,1615,477],{"class":223},[214,1617,1618,1621,1623],{"class":102,"line":284},[214,1619,1620],{"class":223},"          merged.loc[merged.eur_per_m.isna(), ",[214,1622,469],{"class":266},[214,1624,1625],{"class":223},"].unique())\n",[214,1627,1628,1631,1634,1636,1638,1640,1642,1644,1647,1649,1651],{"class":102,"line":312},[214,1629,1630],{"class":223},"merged[",[214,1632,1633],{"class":266},"\"renewal_eur\"",[214,1635,893],{"class":223},[214,1637,260],{"class":219},[214,1639,1576],{"class":223},[214,1641,498],{"class":266},[214,1643,893],{"class":223},[214,1645,1646],{"class":219},"*",[214,1648,1576],{"class":223},[214,1650,1579],{"class":266},[214,1652,276],{"class":223},[214,1654,1655,1657,1660,1662,1664,1666],{"class":102,"line":329},[214,1656,507],{"class":273},[214,1658,1659],{"class":223},"(merged.groupby(",[214,1661,488],{"class":266},[214,1663,270],{"class":223},[214,1665,1633],{"class":266},[214,1667,1668],{"class":223},"].sum().sort_values().tail())\n",[14,1670,1671,1673,1674,1677,1678,746],{},[522,1672,524],{}," Resetting the index before the merge and restoring it afterwards keeps feature ids aligned — a plain ",[186,1675,1676],{},"merge"," drops the index. A left join keeps every pipe, and checking for missing costs immediately catches materials spelled differently in the two sources. For joins that should live in the QGIS project and update automatically, use a layer join instead, as in ",[21,1679,1681],{"href":1680},"\u002Fspatial-data-processing-automation\u002Fattribute-tables-and-field-management\u002Fjoin-attributes-by-field-value-pyqgis\u002F","joining attributes by field value",[174,1683,1685],{"id":1684},"write-computed-columns-back-to-the-layer","Write computed columns back to the layer",[14,1687,1688],{},"Per-feature results — a renewal cost, a risk score, a priority rank — are most useful on the map. Because the frame is indexed by feature id, writing back is a single dictionary comprehension.",[14,1690,1691],{},[29,1692,1695,1698,1701,1704,1711,1714,1716,1721,1725,1728,1734,1738,1741,1745,1748,1752,1755,1760],{"viewBox":1693,"role":32,"ariaLabel":1694,"xmlns":34},"0 0 760 186","A pandas Series indexed by feature id converted to a dictionary and written to the layer in one changeAttributeValues call",[36,1696,1697],{},"Writing back by feature id",[40,1699,1700],{},"A computed pandas Series indexed by feature id is turned into a dictionary mapping each feature id to a dictionary of field index and value. One changeAttributeValues call writes all of them. Missing values become NULL. The new field can then be styled with a graduated renderer.",[44,1702],{"x":46,"y":46,"width":47,"height":1703,"fill":49},"186",[51,1705,1706],{},[54,1707,1709],{"id":1708,"viewBox":57,"refX":58,"refY":59,"markerWidth":60,"markerHeight":60,"orient":61},"pdWriteArrow",[63,1710],{"d":65,"fill":66},[68,1712,1713],{"x":70,"y":71,"style":72,"fill":73,"textAnchor":74},"Series → dict → one provider call",[44,1715],{"x":78,"y":790,"width":840,"height":135,"rx":58,"fill":114,"stroke":115,"style":84},[68,1717,1720],{"x":1718,"y":1719,"style":89,"fill":115,"textAnchor":74},"124","92.78","Series",[68,1722,1724],{"x":1718,"y":1723,"style":94,"fill":66,"textAnchor":74},"118.78","index = fid",[68,1726,1727],{"x":1718,"y":93,"style":94,"fill":95,"textAnchor":74},"renewal_eur",[102,1729],{"x1":1730,"y1":1731,"x2":1732,"y2":1731,"stroke":66,"style":1733},"224","115","266","stroke-width:1.8;marker-end:url(#pdWriteArrow)",[44,1735],{"x":1736,"y":790,"width":1737,"height":135,"rx":58,"fill":148,"stroke":149,"style":84},"270","220",[68,1739,1740],{"x":70,"y":1719,"style":89,"fill":149,"textAnchor":74},"dict",[68,1742,1744],{"x":70,"y":1723,"style":1743,"fill":66,"textAnchor":74},"text-anchor:middle;font-size:10.0px;font-family:monospace","{fid: {idx: value}}",[68,1746,1747],{"x":70,"y":93,"style":94,"fill":95,"textAnchor":74},"NaN → None",[102,1749],{"x1":1750,"y1":1731,"x2":1751,"y2":1731,"stroke":66,"style":1733},"490","532",[44,1753],{"x":1754,"y":790,"width":840,"height":135,"rx":58,"fill":162,"stroke":163,"style":84},"536",[68,1756,1759],{"x":1757,"y":1758,"style":864,"fill":167,"textAnchor":74},"636","105.6","changeAttributeValues",[68,1761,1763],{"x":1757,"y":1762,"style":851,"fill":95,"textAnchor":74},"131.6","one call",[205,1765,1767],{"className":207,"code":1766,"language":209,"meta":210,"style":210},"from qgis.core import QgsField\nfrom qgis.PyQt.QtCore import QVariant\n\nprov = pipes.dataProvider()\nif pipes.fields().indexOf(\"renewal_eur\") \u003C 0:\n    prov.addAttributes([QgsField(\"renewal_eur\", QVariant.Double)])\n    pipes.updateFields()\nidx = pipes.fields().indexOf(\"renewal_eur\")\n\nvalues = merged[\"renewal_eur\"]\nchanges = {int(fid): {idx: (None if pd.isna(v) else float(v))} for fid, v in values.items()}\nok = prov.changeAttributeValues(changes)\nprint(ok, len(changes), \"features updated\")\n",[186,1768,1769,1780,1792,1796,1806,1825,1835,1840,1853,1857,1870,1913,1923],{"__ignoreMap":210},[214,1770,1771,1773,1775,1777],{"class":102,"line":216},[214,1772,236],{"class":219},[214,1774,239],{"class":223},[214,1776,220],{"class":219},[214,1778,1779],{"class":223}," QgsField\n",[214,1781,1782,1784,1787,1789],{"class":102,"line":233},[214,1783,236],{"class":219},[214,1785,1786],{"class":223}," qgis.PyQt.QtCore ",[214,1788,220],{"class":219},[214,1790,1791],{"class":223}," QVariant\n",[214,1793,1794],{"class":102,"line":247},[214,1795,251],{"emptyLinePlaceholder":250},[214,1797,1798,1801,1803],{"class":102,"line":254},[214,1799,1800],{"class":223},"prov ",[214,1802,260],{"class":219},[214,1804,1805],{"class":223}," pipes.dataProvider()\n",[214,1807,1808,1810,1813,1815,1817,1819,1822],{"class":102,"line":279},[214,1809,675],{"class":219},[214,1811,1812],{"class":223}," pipes.fields().indexOf(",[214,1814,1633],{"class":266},[214,1816,593],{"class":223},[214,1818,1461],{"class":219},[214,1820,1821],{"class":273}," 0",[214,1823,1824],{"class":223},":\n",[214,1826,1827,1830,1832],{"class":102,"line":284},[214,1828,1829],{"class":223},"    prov.addAttributes([QgsField(",[214,1831,1633],{"class":266},[214,1833,1834],{"class":223},", QVariant.Double)])\n",[214,1836,1837],{"class":102,"line":312},[214,1838,1839],{"class":223},"    pipes.updateFields()\n",[214,1841,1842,1845,1847,1849,1851],{"class":102,"line":329},[214,1843,1844],{"class":223},"idx ",[214,1846,260],{"class":219},[214,1848,1812],{"class":223},[214,1850,1633],{"class":266},[214,1852,971],{"class":223},[214,1854,1855],{"class":102,"line":340},[214,1856,251],{"emptyLinePlaceholder":250},[214,1858,1859,1862,1864,1866,1868],{"class":102,"line":346},[214,1860,1861],{"class":223},"values ",[214,1863,260],{"class":219},[214,1865,1576],{"class":223},[214,1867,1633],{"class":266},[214,1869,276],{"class":223},[214,1871,1872,1875,1877,1880,1883,1886,1888,1891,1894,1897,1900,1903,1905,1908,1910],{"class":102,"line":355},[214,1873,1874],{"class":223},"changes ",[214,1876,260],{"class":219},[214,1878,1879],{"class":223}," {",[214,1881,1882],{"class":273},"int",[214,1884,1885],{"class":223},"(fid): {idx: (",[214,1887,299],{"class":273},[214,1889,1890],{"class":219}," if",[214,1892,1893],{"class":223}," pd.isna(v) ",[214,1895,1896],{"class":219},"else",[214,1898,1899],{"class":273}," float",[214,1901,1902],{"class":223},"(v))} ",[214,1904,399],{"class":219},[214,1906,1907],{"class":223}," fid, v ",[214,1909,381],{"class":219},[214,1911,1912],{"class":223}," values.items()}\n",[214,1914,1915,1918,1920],{"class":102,"line":361},[214,1916,1917],{"class":223},"ok ",[214,1919,260],{"class":219},[214,1921,1922],{"class":223}," prov.changeAttributeValues(changes)\n",[214,1924,1925,1927,1930,1932,1935,1938],{"class":102,"line":372},[214,1926,507],{"class":273},[214,1928,1929],{"class":223},"(ok, ",[214,1931,1479],{"class":273},[214,1933,1934],{"class":223},"(changes), ",[214,1936,1937],{"class":266},"\"features updated\"",[214,1939,971],{"class":223},[14,1941,1942,1944,1945,1948,1949,1952,1953,1955,1956,1958],{},[522,1943,524],{}," Casting ids with ",[186,1946,1947],{},"int()"," and values with ",[186,1950,1951],{},"float()"," converts NumPy scalar types into plain Python types the provider accepts. Missing values become ",[186,1954,299],{},", stored as NULL. One ",[186,1957,1759],{}," call writes every row, which on GeoPackage is a single transaction. Add the field only if it does not exist, so the script can run repeatedly. For an undoable change in an interactive session, write through the edit buffer instead.",[174,1960,1962],{"id":1961},"export-tables-for-reports","Export tables for reports",[14,1964,1965],{},"The summaries are usually destined for a spreadsheet or a document. Pandas writes Excel with several sheets in one call.",[205,1967,1969],{"className":207,"code":1968,"language":209,"meta":210,"style":210},"with pd.ExcelWriter(\"\u002Fdata\u002Freports\u002Fpipe_renewal_2026.xlsx\") as xl:\n    by_material.round(2).to_excel(xl, sheet_name=\"by material\")\n    rate.round(2).to_excel(xl, sheet_name=\"rate by age\")\n    (merged.groupby(\"district\")[\"renewal_eur\"].sum().round(0)\n           .to_frame().to_excel(xl, sheet_name=\"cost by district\"))\n",[186,1970,1971,1989,2009,2027,2045],{"__ignoreMap":210},[214,1972,1973,1976,1979,1982,1984,1986],{"class":102,"line":216},[214,1974,1975],{"class":219},"with",[214,1977,1978],{"class":223}," pd.ExcelWriter(",[214,1980,1981],{"class":266},"\"\u002Fdata\u002Freports\u002Fpipe_renewal_2026.xlsx\"",[214,1983,593],{"class":223},[214,1985,227],{"class":219},[214,1987,1988],{"class":223}," xl:\n",[214,1990,1991,1994,1996,1999,2002,2004,2007],{"class":102,"line":233},[214,1992,1993],{"class":223},"    by_material.round(",[214,1995,1414],{"class":273},[214,1997,1998],{"class":223},").to_excel(xl, ",[214,2000,2001],{"class":419},"sheet_name",[214,2003,260],{"class":219},[214,2005,2006],{"class":266},"\"by material\"",[214,2008,971],{"class":223},[214,2010,2011,2014,2016,2018,2020,2022,2025],{"class":102,"line":247},[214,2012,2013],{"class":223},"    rate.round(",[214,2015,1414],{"class":273},[214,2017,1998],{"class":223},[214,2019,2001],{"class":419},[214,2021,260],{"class":219},[214,2023,2024],{"class":266},"\"rate by age\"",[214,2026,971],{"class":223},[214,2028,2029,2032,2034,2036,2038,2041,2043],{"class":102,"line":254},[214,2030,2031],{"class":223},"    (merged.groupby(",[214,2033,488],{"class":266},[214,2035,270],{"class":223},[214,2037,1633],{"class":266},[214,2039,2040],{"class":223},"].sum().round(",[214,2042,46],{"class":273},[214,2044,971],{"class":223},[214,2046,2047,2050,2052,2054,2057],{"class":102,"line":279},[214,2048,2049],{"class":223},"           .to_frame().to_excel(xl, ",[214,2051,2001],{"class":419},[214,2053,260],{"class":219},[214,2055,2056],{"class":266},"\"cost by district\"",[214,2058,444],{"class":223},[14,2060,2061,2063,2064,2067,2068,2071,2072,2075,2076,746],{},[522,2062,524],{}," Each ",[186,2065,2066],{},"to_excel"," call writes one sheet; rounding before export keeps the spreadsheet readable. For CSV, ",[186,2069,2070],{},"to_csv"," with ",[186,2073,2074],{},"encoding=\"utf-8-sig\""," makes Excel open accented characters correctly. If the report also needs maps, a print layout with an attribute table item can show the same numbers next to the map, as in ",[21,2077,2079],{"href":2078},"\u002Fspatial-data-processing-automation\u002Fautomated-map-layout-generation\u002Fadd-attribute-table-to-layout-pyqgis\u002F","adding an attribute table to a layout",[174,2081,2083],{"id":2082},"qgis-version-compatibility","QGIS version compatibility",[14,2085,2086,2087,2089,2090,2093,2094,2097,2098,2101,2102,746],{},"The code works on QGIS 3.34 LTR, 3.40 LTR and QGIS 4 with pandas 1.5 or newer. On QGIS 4, NULLs arrive as ",[186,2088,299],{}," and the cleaning step simply passes them through; ",[186,2091,2092],{},"QgsField"," takes ",[186,2095,2096],{},"QMetaType.Type.Double"," instead of ",[186,2099,2100],{},"QVariant.Double",". Excel export needs ",[186,2103,195],{},[174,2105,2107],{"id":2106},"troubleshooting","Troubleshooting",[179,2109,2110,2116,2122,2139],{},[182,2111,2112,2115],{},[522,2113,2114],{},"Numeric columns have object dtype."," NULL QVariants were not cleaned; run the cleaning step.",[182,2117,2118,2121],{},[522,2119,2120],{},"Writing back changes the wrong features."," The index was lost in a merge; reset and restore it as shown.",[182,2123,2124,2132,2133,2135,2136,746],{},[522,2125,2126,2129,2130,746],{},[186,2127,2128],{},"TypeError"," on ",[186,2131,1759],{}," NumPy types were passed; cast to ",[186,2134,1882],{}," and ",[186,2137,2138],{},"float",[182,2140,2141,2144],{},[522,2142,2143],{},"Rates look wrong."," Per-feature rates were averaged; aggregate totals first, then divide.",[174,2146,2148],{"id":2147},"conclusion","Conclusion",[14,2150,2151],{},"Read attributes without geometry into a DataFrame indexed by feature id, clean NULLs and Qt dates once, summarise with group-bys and pivots on aggregated totals, merge external tables without losing the index, write computed columns back in one provider call, and export summaries straight to Excel.",[174,2153,2155],{"id":2154},"frequently-asked-questions","Frequently Asked Questions",[14,2157,2158,2161,2162,2165],{},[522,2159,2160],{},"Is this faster than QGIS aggregate expressions?","\nFor one aggregate, ",[186,2163,2164],{},"layer.aggregate"," is fine. For many group-bys, pivots and merges, pandas is far quicker to write and to run.",[14,2167,2168,2171],{},[522,2169,2170],{},"Can I use polars instead of pandas?","\nYes. Build the frame from the same rows; the write-back pattern is identical.",[14,2173,2174,2177],{},[522,2175,2176],{},"Should I store results in the layer or a separate table?","\nPer-feature values that drive styling belong in the layer; summaries belong in a table or report.",[14,2179,2180,2183,2184,2187],{},[522,2181,2182],{},"What about very large tables?","\nRead in chunks with ",[186,2185,2186],{},"setLimit"," and keyset paging, or query the database directly with SQL.",[174,2189,2191],{"id":2190},"related","Related",[179,2193,2194,2199,2205,2211,2217],{},[182,2195,2196,2198],{},[21,2197,24],{"href":23}," — the guide this recipe belongs to",[182,2200,2201],{},[21,2202,2204],{"href":2203},"\u002Fspatial-data-processing-automation\u002Fpyqgis-and-the-python-data-stack\u002Fplot-layer-data-with-matplotlib-pyqgis\u002F","Plot Layer Data with Matplotlib in PyQGIS",[182,2206,2207],{},[21,2208,2210],{"href":2209},"\u002Fspatial-data-processing-automation\u002Fattribute-tables-and-field-management\u002Fexport-attribute-table-to-csv-pyqgis\u002F","Export an Attribute Table to CSV in PyQGIS",[182,2212,2213],{},[21,2214,2216],{"href":2215},"\u002Fpyqgis-fundamentals-environment-setup\u002Fworking-with-qgis-expressions\u002Fuse-aggregate-expressions-pyqgis\u002F","Use Aggregate Expressions in PyQGIS",[182,2218,2219],{},[21,2220,2222],{"href":2221},"\u002Fspatial-data-processing-automation\u002Fattribute-tables-and-field-management\u002Fcount-features-by-attribute-pyqgis\u002F","Count Features by Attribute in PyQGIS",[2224,2225,2226],"style",{},"html pre.shiki code .snl16, html code.shiki .snl16{--shiki-default:#F97583}html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .s9osk, html code.shiki .s9osk{--shiki-default:#FFAB70}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .sjoCn, html code.shiki .sjoCn{--shiki-default:#9AA79F}",{"title":210,"searchDepth":233,"depth":233,"links":2228},[2229,2230,2231,2232,2233,2234,2235,2236,2237,2238,2239,2240,2241],{"id":176,"depth":233,"text":177},{"id":199,"depth":233,"text":200},{"id":531,"depth":233,"text":532},{"id":749,"depth":233,"text":750},{"id":1217,"depth":233,"text":1218},{"id":1503,"depth":233,"text":1504},{"id":1684,"depth":233,"text":1685},{"id":1961,"depth":233,"text":1962},{"id":2082,"depth":233,"text":2083},{"id":2106,"depth":233,"text":2107},{"id":2147,"depth":233,"text":2148},{"id":2154,"depth":233,"text":2155},{"id":2190,"depth":233,"text":2191},"Load a layer's attribute table into a pandas DataFrame without geometry, summarise it with group-bys and pivot tables, join it to spreadsheets, write computed columns back to the layer by feature id, and export tidy tables for reports.","md",{"slug":2245,"type":2246,"breadcrumb":2247,"datePublished":2248,"dateModified":2248},"analyse-attribute-table-with-pandas-pyqgis","article","Analyse an Attribute Table with Pandas","2026-10-02","\u002Fspatial-data-processing-automation\u002Fpyqgis-and-the-python-data-stack\u002Fanalyse-attribute-table-with-pandas-pyqgis",{"title":5,"description":2242},"spatial-data-processing-automation\u002Fpyqgis-and-the-python-data-stack\u002Fanalyse-attribute-table-with-pandas-pyqgis\u002Findex","Xmbkjvrs8VGSgQyXqmM5O-lszloR--7OWbt2TyMuJfE",1790966264325]