API Reference (automated)
_helpers
- _helpers.configure_logging(snakemake, skip_handlers=False)
Configure the basic behaviour for the logging module.
Note: Must only be called once from the __main__ section of a script.
The setup includes printing log messages to STDERR and to a log file defined by either (in priority order): snakemake.log.python, snakemake.log[0] or “logs/{rulename}.log”. Additional keywords from logging.basicConfig are accepted via the snakemake configuration file under snakemake.config.logging.
- Parameters
snakemake (snakemake object) – Your snakemake object containing a snakemake.config and snakemake.log.
skip_handlers (True | False (default)) – Do (not) skip the default handlers created for redirecting output to STDERR and file.
- _helpers.country_name_2_two_digits(country_name)
Convert full country name to 2-digit country code
- _helpers.getContinent(code)
Returns continent names that contains list of iso-code countries
- Parameters
code (str) – List of two letter country ISO codes
- Returns
continent_list – List of continent names
- Return type
Example
from helpers import getContinent code = [“DE”, “GB”, “NG”, “ZA”] getContinent(code) >>> [“africa”, “europe”]
- _helpers.get_aggregation_strategies(aggregation_strategies)
default aggregation strategies that cannot be defined in .yaml format must be specified within the function, otherwise (when defaults are passed in the function’s definition) they get lost when custom values are specified in the config.
- _helpers.get_country(target, **keys)
Function to convert country codes using pycountry
- Parameters
target (str) –
Desired type of country code. Examples:
’alpha_3’ for 3-digit
’alpha_2’ for 2-digit
’name’ for full country name
keys (dict) –
Specification of the country name and reference system. Examples:
alpha_3=”ZAF” for 3-digit
alpha_2=”ZA” for 2-digit
name=”South Africa” for full country name
- Returns
country code as requested in keys or np.nan, when country code is not recognized
Example of usage
——-
- Convert 2-digit code to 3-digit codes (get_country(‘alpha_3’, alpha_2=”ZA”))
- Convert 3-digit code to 2-digit codes (get_country(‘alpha_2’, alpha_3=”ZAF”))
- Convert 2-digit code to full name (get_country(‘name’, alpha_2=”ZA”))
- _helpers.load_network(import_name=None, custom_components=None)
Helper for importing a pypsa.Network with additional custom components.
- Parameters
import_name (str) – As in pypsa.Network(import_name)
custom_components (dict) –
Dictionary listing custom components. For using
snakemake.config["override_components"]
inconfig.yaml
define:override_components: ShadowPrice: component: ["shadow_prices","Shadow price for a global constraint.",np.nan] attributes: name: ["string","n/a","n/a","Unique name","Input (required)"] value: ["float","n/a",0.,"shadow value","Output"]
- Return type
pypsa.Network
- _helpers.mock_snakemake(rulename, **wildcards)
This function is expected to be executed from the “scripts”-directory of ” the snakemake project. It returns a snakemake.script.Snakemake object, based on the Snakefile.
If a rule has wildcards, you have to specify them in **wildcards.
- Parameters
rulename (str) – name of the rule for which the snakemake object should be generated
**wildcards – keyword arguments fixing the wildcards. Only necessary if wildcards are needed.
- _helpers.progress_retrieve(url, file, data=None, disable_progress=False, roundto=1.0)
Function to download data from a url with a progress bar progress in retrieving data
- Parameters
url (str) – Url to download data from
file (str) – File where to save the output
data (dict) – Data for the request (default None), when not none Post method is used
disable_progress (bool) – When true, no progress bar is shown
roundto (float) – (default 0) Precision used to report the progress e.g. 0.1 stands for 88.1, 10 stands for 90, 80
- _helpers.read_csv_nafix(file, **kwargs)
Function to open a csv as pandas file and standardize the na value
- _helpers.sets_path_to_root(root_directory_name)
Search and sets path to the given root directory (root/path/file).
- _helpers.three_2_two_digits_country(three_code_country)
Convert 3-digit to 2-digit country code:
- _helpers.two_2_three_digits_country(two_code_country)
Convert 2-digit to 3-digit country code:
- _helpers.two_digits_2_name_country(two_code_country, nocomma=False, remove_start_words=[])
Convert 2-digit country code to full name country:
- Parameters
two_code_country (str) – 2-digit country name
nocomma (bool (optional, default False)) – When true, country names with comma are extended to remove the comma. Example CD -> Congo, The Democratic Republic of -> The Democratic Republic of Congo
remove_start_words (list (optional, default empty)) – When a sentence starts with any of the provided words, the beginning is removed. e.g. The Democratic Republic of Congo -> Democratic Republic of Congo (remove_start_words=[“The”])
- Returns
full_name – full country name
- Return type
add_electricity
Adds electrical generators, load and existing hydro storage units to a base network.
Relevant Settings
costs:
year:
USD2013_to_EUR2013:
dicountrate:
emission_prices:
electricity:
max_hours:
marginal_cost:
capital_cost:
conventional_carriers:
co2limit:
extendable_carriers:
include_renewable_capacities_from_OPSD:
estimate_renewable_capacities_from_capacity_stats:
load:
scale:
ssp:
weather_year:
prediction_year:
region_load:
renewable:
hydro:
carriers:
hydro_max_hours:
hydro_capital_cost:
lines:
length_factor:
See also
Documentation of the configuration file config.yaml
at costs,
electricity, load, renewable, lines
Inputs
data/costs.csv
: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.data/bundle/hydro_capacities.csv
: Hydropower plant store/discharge power capacities, energy storage capacity, and average hourly inflow by country. Not currently used!data/geth2015_hydro_capacities.csv
: alternative to capacities above; not currently used!resources/ssp2-2.6/2030/era5_2013/Africa.nc
Hourly country load profiles produced by GEGISresources/regions_onshore.geojson
: confer Rule build_bus_regionsresources/gadm_shapes.geojson
: confer Rule build_shapesresources/powerplants.csv
: confer Rule build_powerplantsresources/profile_{}.nc
: all technologies inconfig["renewables"].keys()
, confer Rule build_renewable_profiles.networks/base.nc
: confer Rule base_network
Outputs
networks/elec.nc
:
Description
The rule add_electricity
ties all the different data inputs from the preceding rules together into a detailed PyPSA network that is stored in networks/elec.nc
. It includes:
today’s transmission topology and transfer capacities (in future, optionally including lines which are under construction according to the config settings
lines: under_construction
andlinks: under_construction
),today’s thermal and hydro power generation capacities (for the technologies listed in the config setting
electricity: conventional_carriers
), andtoday’s load time-series (upsampled in a top-down approach according to population and gross domestic product)
It further adds extendable generators
with zero capacity for
photovoltaic, onshore and AC- as well as DC-connected offshore wind installations with today’s locational, hourly wind and solar capacity factors (but no current capacities),
additional open- and combined-cycle gas turbines (if
OCGT
and/orCCGT
is listed in the config settingelectricity: extendable_carriers
)
- add_electricity.attach_load(n, load_paths, regions, admin_shapes, countries, scale)
Add load to the network and distributes them according GDP and population.
- Parameters
n (pypsa network) –
regions (.geojson) – Contains bus_id of low voltage substations and bus region shapes (voronoi cells)
load_paths (paths of the load files) –
admin_shapes (.geojson) – contains subregional gdp, population and shape data
countries (list) – List of countries that is config input
scale (float) – The scale factor is multiplied with the load (1.3 = 30% more load)
- Returns
n – Now attached with load time series
- Return type
pypsa network
- add_electricity.calculate_annuity(n, r)
Calculate the annuity factor for an asset with lifetime n years and discount rate of r, e.g. annuity(20, 0.05) * 20 = 1.6
- add_electricity.get_load_paths_gegis(ssp_parentfolder, config)
Creates load paths for the GEGIS outputs
The paths are created automatically according to included country, weather year, prediction year and ssp scenario
Example
[“/data/ssp2-2.6/2030/era5_2013/Africa.nc”, “/data/ssp2-2.6/2030/era5_2013/Africa.nc”]
- add_electricity.load_costs(tech_costs, config, elec_config, Nyears=1)
set all asset costs and other parameters
base_network
Creates the network topology from a OpenStreetMap
Relevant Settings
snapshots:
countries:
electricity:
voltages:
lines:
types:
s_max_pu:
under_construction:
links:
p_max_pu:
under_construction:
include_tyndp:
transformers:
x:
s_nom:
type:
See also
Documentation of the configuration file config.yaml
at
snapshots, Top-level configuration, electricity, load,
lines, links, transformers
Inputs
Outputs
networks/base.nc
Description
build_bus_regions
Creates Voronoi shapes for each bus representing both onshore and offshore regions.
Relevant Settings
countries:
See also
Documentation of the configuration file config.yaml
at
Top-level configuration
Inputs
resources/country_shapes.geojson
: confer Rule build_shapesresources/offshore_shapes.geojson
: confer Rule build_shapesnetworks/base.nc
: confer Rule base_network
Outputs
resources/regions_onshore.geojson
:resources/regions_offshore.geojson
:
Description
- build_bus_regions.custom_voronoi_partition_pts(points, outline, add_bounds_shape=True, multiplier=5)
Compute the polygons of a voronoi partition of points within the polygon outline
- build_bus_regions.points
- Type
Nx2 - ndarray[dtype=float]
- build_bus_regions.outline
- Type
Polygon
- build_bus_regions.no_multipolygons
If true, replace each MultiPolygon by its largest component
- Type
bool (default: False)
- Returns
polygons
- Return type
N - ndarray[dtype=Polygon|MultiPolygon]
build_cutout
Create cutouts with atlite.
For this rule to work you must have
installed the Copernicus Climate Data Store
cdsapi
package (install with `pip`) andregistered and setup your CDS API key as described on their website. The CDS API allows an automatic filedownload by executing this script
See also
For details on the weather data read the atlite documentation. If you need help specifically for creating cutouts the corresponding section in the atlite documentation should be helpful.
Relevant Settings
atlite:
nprocesses:
cutouts:
{cutout}:
See also
Documentation of the configuration file config.yaml
at
atlite
Inputs
None
Outputs
cutouts/{cutout}
: weather data from either the ERA5 reanalysis weather dataset or SARAH-2 satellite-based historic weather data with the following structure:
ERA5 cutout:
Field
Dimensions
Unit
Description
pressure
time, y, x
Pa
Surface pressure
temperature
time, y, x
K
Air temperature 2 meters above the surface.
soil temperature
time, y, x
K
Soil temperature between 1 meters and 3 meters depth (layer 4).
influx_toa
time, y, x
Wm**-2
Top of Earth’s atmosphere TOA incident solar radiation
influx_direct
time, y, x
Wm**-2
Total sky direct solar radiation at surface
runoff
time, y, x
m
Runoff (volume per area)
roughness
y, x
m
Forecast surface roughness (roughness length)
height
y, x
m
Surface elevation above sea level
albedo
time, y, x
–
Albedo measure of diffuse reflection of solar radiation. Calculated from relation between surface solar radiation downwards (Jm**-2) and surface net solar radiation (Jm**-2). Takes values between 0 and 1.
influx_diffuse
time, y, x
Wm**-2
Diffuse solar radiation at surface. Surface solar radiation downwards minus direct solar radiation.
wnd100m
time, y, x
ms**-1
Wind speeds at 100 meters (regardless of direction)
![]()
A SARAH-2 cutout can be used to amend the fields temperature
, influx_toa
, influx_direct
, albedo
,
influx_diffuse
of ERA5 using satellite-based radiation observations.
Description
build_natura_raster
Converts vectordata or known as shapefiles (i.e. used for geopandas/shapely) to our cutout rasters. The Protected Planet Data on protected areas is aggregated to all cutout regions.
Relevant Settings
renewable:
{technology}:
cutout:
See also
Documentation of the configuration file config.yaml
at
renewable
Inputs
data/raw/protected_areas/WDPA_WDOECM_Aug2021_Public_AF_shp-points.shp
: WDPA World Database for Protected Areas.
Outputs
resources/natura.tiff
: Rasterized version of Natura 2000 natural protection areas to reduce computation times.
Description
To operate the script you need all input files. The Snakefile describes what goes in and out. Make sure you didn’t skip one of these. Maybe not so obvious is the cutout input. An example is this africa-2013-era5.nc
- Steps to retrieve the protected area data (as apparently no API is given for the WDPA data):
Download the WPDA Dataset: World Database on Protected Areas. UNEP-WCMC and IUCN (2021), Protected Planet: The World Database on Protected Areas (WDPA) and World Database on Other Effective Area-based Conservation Measures (WD-OECM) [Online], August 2021, Cambridge, UK: UNEP-WCMC and IUCN. Available at: www.protectedplanet.net.
Unzipp and rename the folder containing the .shp file to protected_areas
Important! Don’t delete the other files which come with the .shp file. They are required to build the shape.
Move the file in such a way that the above path is given
Activate the environment of environment-max.yaml
Ready to run the script
Tip: The output file natura.tiff contains now the 100x100m rasters of protective areas. This operation can make the filesize of that TIFF quite large and leads to problems when trying to open. QGIS, an open source tool helps exploring the file.
- build_natura_raster.get_fileshapes(list_paths, accepted_formats=('.shp',))
Function to parse the list of paths to include shapes included in folders, if any
- build_natura_raster.unify_protected_shape_areas(inputs, area_crs, out_logging)
Iterates thorugh all snakemake rule inputs and unifies shapefiles (.shp) only.
The input is given in the Snakefile and shapefiles are given by .shp
- Returns
unified_shape
- Return type
GeoDataFrame with a unified “multishape”
build_renewable_profiles
Calculates for each network node the (i) installable capacity (based on land-use), (ii) the available generation time series (based on weather data), and (iii) the average distance from the node for onshore wind, AC-connected offshore wind, DC-connected offshore wind and solar PV generators. For hydro generators, it calculates the expected inflows. In addition for offshore wind it calculates the fraction of the grid connection which is under water.
Relevant settings
snapshots:
atlite:
nprocesses:
renewable:
{technology}:
cutout:
copernicus:
grid_codes:
distance:
distance_grid_codes:
natura:
max_depth:
max_shore_distance:
min_shore_distance:
capacity_per_sqkm:
correction_factor:
potential:
min_p_max_pu:
clip_p_max_pu:
resource:
clip_min_inflow:
Inputs
data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_EPSG-4326.tif
: Copernicus Land Service inventory on 23 land use classes (e.g. forests, arable land, industrial, urban areas) based on UN-FAO classification. See Table 4 in the PUM for a list of all classes.data/gebco/GEBCO_2021_TID.nc
: A bathymetric data set with a global terrain model for ocean and land at 15 arc-second intervals by the General Bathymetric Chart of the Oceans (GEBCO).Source: GEBCO
resources/natura.tiff
: confer Rule build_natura_rasterresources/offshore_shapes.geojson
: confer Rule build_shapesresources/.geojson
: (if not offshore wind), confer Rule build_bus_regionsresources/regions_offshore.geojson
: (if offshore wind), Rule build_bus_regions"cutouts/" + config["renewable"][{technology}]['cutout']
: Rule build_cutoutnetworks/base.nc
: Rule base_network
Outputs
resources/profile_{technology}.nc
, except hydro technology, with the following structureField
Dimensions
Description
profile
bus, time
the per unit hourly availability factors for each node
weight
bus
sum of the layout weighting for each node
p_nom_max
bus
maximal installable capacity at the node (in MW)
potential
y, x
layout of generator units at cutout grid cells inside the Voronoi cell (maximal installable capacity at each grid cell multiplied by capacity factor)
average_distance
bus
average distance of units in the Voronoi cell to the grid node (in km)
underwater_fraction
bus
fraction of the average connection distance which is under water (only for offshore)
Description
This script leverages on atlite function to derivate hourly time series for an entire year for solar, wind (onshore and offshore), and hydro data.
This script functions at two main spatial resolutions: the resolution of the network nodes and their Voronoi cells, and the resolution of the cutout grid cells for the weather data. Typically the weather data grid is finer than the network nodes, so we have to work out the distribution of generators across the grid cells within each Voronoi cell. This is done by taking account of a combination of the available land at each grid cell and the capacity factor there.
This uses the Copernicus land use data, Natura2000 nature reserves and GEBCO bathymetry data.

To compute the layout of generators in each node’s Voronoi cell, the installable potential in each grid cell is multiplied with the capacity factor at each grid cell. This is done since we assume more generators are installed at cells with a higher capacity factor.




This layout is then used to compute the generation availability time series
from the weather data cutout from atlite
.
Two methods are available to compute the maximal installable potential for the
node (p_nom_max): simple
and conservative
:
simple
adds up the installable potentials of the individual grid cells. If the model comes close to this limit, then the time series may slightly overestimate production since it is assumed the geographical distribution is proportional to capacity factor.conservative
assertains the nodal limit by increasing capacities proportional to the layout until the limit of an individual grid cell is reached.
build_shapes
- build_shapes.add_gdp_data(df_gadm, year=2020, update=False, out_logging=False, name_file_nc='GDP_PPP_1990_2015_5arcmin_v2.nc', nprocesses=2, disable_progressbar=False)
Function to add gdp data to arbitrary number of shapes in a country
- df_gadm: Geodataframe with one Multipolygon per row
Essential column [“country”, “geometry”]
Non-essential column [“GADM_ID”]
- df_gadm: Geodataframe with one Multipolygon per row
Same columns as input
Includes a new column [“gdp”]
- build_shapes.add_population_data(df_gadm, country_codes, worldpop_method, year=2020, update=False, out_logging=False, nprocesses=2, disable_progressbar=False)
Function to add population data to arbitrary number of shapes in a country
- df_gadm: Geodataframe with one Multipolygon per row
Essential column [“country”, “geometry”]
Non-essential column [“GADM_ID”]
- df_gadm: Geodataframe with one Multipolygon per row
Same columns as input
Includes a new column [“pop”]
- build_shapes.convert_GDP(name_file_nc, year=2015, out_logging=False)
Function to convert the nc database of the GDP to tif, based on the work at https://doi.org/10.1038/sdata.2018.4. The dataset shall be downloaded independently by the user (see guide) or toghether with pypsa-africa package.
- build_shapes.countries(countries, geo_crs, update=False, out_logging=False)
Create country shapes
- build_shapes.download_GADM(country_code, update=False, out_logging=False)
Download gpkg file from GADM for a given country code
- build_shapes.download_WorldPop(country_code, worldpop_method, year=2020, update=False, out_logging=False, size_min=300)
- Download Worldpop using either the standard method or the API method.
- worldpop_method: str
worldpop_method = “api” will use the API method to access the WorldPop 100mx100m dataset. worldpop_method = “standard” will use the standard method to access the WorldPop 1KMx1KM dataset.
- country_codestr
Two letter country codes of the downloaded files. Files downloaded from https://data.worldpop.org/ datasets WorldPop UN adjusted
- yearint
Year of the data to download
- updatebool
Update = true, forces re-download of files
- size_minint
Minimum size of each file to download
- build_shapes.download_WorldPop_API(country_code, year=2020, update=False, out_logging=False, size_min=300)
Download tiff file for each country code using the api method from worldpop API with 100mx100m resolution.
- Parameters
country_code (str) – Two letter country codes of the downloaded files. Files downloaded from https://data.worldpop.org/ datasets WorldPop UN adjusted
year (int) – Year of the data to download
update (bool) – Update = true, forces re-download of files
size_min (int) – Minimum size of each file to download
- Returns
WorldPop_inputfile (str) – Path of the file
WorldPop_filename (str) – Name of the file
- build_shapes.download_WorldPop_standard(country_code, year=2020, update=False, out_logging=False, size_min=300)
Download tiff file for each country code using the standard method from worldpop datastore with 1kmx1km resolution.
- Parameters
country_code (str) – Two letter country codes of the downloaded files. Files downloaded from https://data.worldpop.org/ datasets WorldPop UN adjusted
year (int) – Year of the data to download
update (bool) – Update = true, forces re-download of files
size_min (int) – Minimum size of each file to download
- Returns
WorldPop_inputfile (str) – Path of the file
WorldPop_filename (str) – Name of the file
- build_shapes.eez(countries, geo_crs, country_shapes, EEZ_gpkg, out_logging=False, distance=0.01)
Creates offshore shapes by - buffer smooth countryshape (=offset country shape) - and differ that with the offshore shape Leads to for instance a 100m non-build coastline
- build_shapes.generalized_mask(src, geom, **kwargs)
Generalize mask function to account for Polygon and MultiPolygon
- build_shapes.get_GADM_layer(country_list, layer_id, geo_crs, update=False, outlogging=False)
Function to retrive a specific layer id of a geopackage for a selection of countries
- build_shapes.load_EEZ(countries_codes, geo_crs, EEZ_gpkg='./data/eez/eez_v11.gpkg')
Function to load the database of the Exclusive Economic Zones. The dataset shall be downloaded independently by the user (see guide) or together with pypsa-africa package.
- build_shapes.load_GDP(countries_codes, year=2015, update=False, out_logging=False, name_file_nc='GDP_PPP_1990_2015_5arcmin_v2.nc')
Function to load the database of the GDP, based on the work at https://doi.org/10.1038/sdata.2018.4. The dataset shall be downloaded independently by the user (see guide) or toghether with pypsa-africa package.
cluster_network
Creates networks clustered to {cluster}
number of zones with aggregated buses, generators and transmission corridors.
Relevant Settings
clustering:
aggregation_strategies:
focus_weights:
solving:
solver:
name:
lines:
length_factor:
See also
Documentation of the configuration file config.yaml
at
Top-level configuration, renewable, solving, lines
Inputs
resources/regions_onshore_elec_s{simpl}.geojson
: confer Rule simplify_networkresources/regions_offshore_elec_s{simpl}.geojson
: confer Rule simplify_networkresources/busmap_elec_s{simpl}.csv
: confer Rule simplify_networknetworks/elec_s{simpl}.nc
: confer Rule simplify_networkdata/custom_busmap_elec_s{simpl}_{clusters}.csv
: optional input
Outputs
resources/regions_onshore_elec_s{simpl}_{clusters}.geojson
:resources/regions_offshore_elec_s{simpl}_{clusters}.geojson
:resources/busmap_elec_s{simpl}_{clusters}.csv
: Mapping of buses fromnetworks/elec_s{simpl}.nc
tonetworks/elec_s{simpl}_{clusters}.nc
;resources/linemap_elec_s{simpl}_{clusters}.csv
: Mapping of lines fromnetworks/elec_s{simpl}.nc
tonetworks/elec_s{simpl}_{clusters}.nc
;networks/elec_s{simpl}_{clusters}.nc
:
Description
Note
Why is clustering used both in simplify_network
and cluster_network
?
Consider for example a network
networks/elec_s100_50.nc
in whichsimplify_network
clusters the network to 100 buses and in a second stepcluster_network`
reduces it down to 50 buses.In preliminary tests, it turns out, that the principal effect of changing spatial resolution is actually only partially due to the transmission network. It is more important to differentiate between wind generators with higher capacity factors from those with lower capacity factors, i.e. to have a higher spatial resolution in the renewable generation than in the number of buses.
The two-step clustering allows to study this effect by looking at networks like
networks/elec_s100_50m.nc
. Note the additionalm
in the{cluster}
wildcard. So in the example network there are still up to 100 different wind generators.In combination these two features allow you to study the spatial resolution of the transmission network separately from the spatial resolution of renewable generators.
Is it possible to run the model without the simplify_network
rule?
No, the network clustering methods in the PyPSA module pypsa.networkclustering do not work reliably with multiple voltage levels and transformers.
Tip
The rule cluster_all_networks
runs
for all scenario
s in the configuration file
the rule cluster_network
.
Exemplary unsolved network clustered to 512 nodes:

Exemplary unsolved network clustered to 256 nodes:

Exemplary unsolved network clustered to 128 nodes:

Exemplary unsolved network clustered to 37 nodes:

- cluster_network.distribute_clusters(n, n_clusters, focus_weights=None, solver_name=None)
Determine the number of clusters per country
build_osm_network
- build_osm_network.connect_stations_same_station_id(lines, buses)
Function to create fake links between substations with the same substation_id
- build_osm_network.fix_overpassing_lines(lines, buses, distance_crs, tol=1)
Function to avoid buses overpassing lines with no connection when the bus is within a given tolerance from the line
- Parameters
lines (GeoDataFrame) – Geodataframe of lines
buses (GeoDataFrame) – Geodataframe of substations
tol (float) – Tolerance in meters of the distance between the substation and the line below which the line will be splitted
- build_osm_network.get_ac_frequency(df, fr_col='tag_frequency')
# Function to define a default frequency value. Attempts to find the most usual non-zero frequency across the dataframe; 50 Hz is assumed as a back-up value
- build_osm_network.get_converters(buses, lines)
Function to create fake converter lines that connect buses of the same station_id of different polarities
- build_osm_network.get_transformers(buses, lines)
Function to create fake transformer lines that connect buses of the same station_id at different voltage
- build_osm_network.merge_stations_lines_by_station_id_and_voltage(lines, buses, geo_crs, distance_crs, tol=2000)
Function to merge close stations and adapt the line datasets to adhere to the merged dataset
- build_osm_network.merge_stations_same_station_id(buses, delta_lon=0.001, delta_lat=0.001, precision=4)
Function to merge buses with same voltage and station_id This function iterates over all substation ids and creates a bus_id for every substation and voltage level. Therefore, a substation with multiple voltage levels is represented with different buses, one per voltage level
- build_osm_network.set_lines_ids(lines, buses, distance_crs)
Function to set line buses ids to the closest bus in the list
- build_osm_network.set_lv_substations(buses)
Function to set what nodes are lv, thereby setting substation_lv The current methodology is to set lv nodes to buses where multiple voltage level are found, hence when the station_id is duplicated
- build_osm_network.set_substations_ids(buses, distance_crs, tol=2000)
Function to set substations ids to buses, accounting for location tolerance
The algorithm is as follows:
initialize all substation ids to -1
if the current substation has been already visited [substation_id < 0], then skip the calculation
- otherwise:
identify the substations within the specified tolerance (tol)
when all the substations in tolerance have substation_id < 0, then specify a new substation_id
otherwise, if one of the substation in tolerance has a substation_id >= 0, then set that substation_id to all the others; in case of multiple substations with substation_ids >= 0, the first value is picked for all
clean_osm_data
config_osm_data
download_osm_data
OSM extraction script.
- download_osm_data.convert_filtered_data_to_dfs(country_code, feature_data, feature)
Convert Filtered Data, Elements to Pandas Dataframes
- download_osm_data.convert_iso_to_geofk(iso_code, iso_coding=True, convert_dict={'AE': 'GCC', 'BH': 'GCC', 'BN': 'MY-SG-BN', 'EH': 'MA', 'GM': 'SNGM', 'HK': 'CN', 'IL': 'IL-PL', 'KW': 'GCC', 'MO': 'CN', 'MY': 'MY-SG-BN', 'OM': 'GCC', 'PS': 'IL-PL', 'QA': 'GCC', 'SA': 'GCC', 'SG': 'MY-SG-BN', 'SM': 'IT', 'SN': 'SNGM'})
Function to convert the iso code name of a country into the corresponding geofabrik In Geofabrik, some countries are aggregated, thus if a single country is requested, then all the agglomeration shall be downloaded For example, Senegal (SN) and Gambia (GM) cannot be found alone in geofabrik, but they can be downloaded as a whole SNGM
The conversion directory, initialized to iso_to_geofk_dict is used to perform such conversion When a two-letter code country is found in convert_dict, and iso_coding is enabled, then that two-letter code is converted into the corresponding value of the dictionary
- download_osm_data.convert_pd_to_gdf_lines(df_way, geo_crs, simplified=False)
Convert Lines Pandas Dataframe to GeoPandas Dataframe
- download_osm_data.convert_pd_to_gdf_nodes(df_way, geo_crs)
Convert Points Pandas Dataframe to GeoPandas Dataframe
- download_osm_data.convert_ways_lines(df_way, Data, geo_crs, distance_crs)
Convert Ways to Line Coordinates
- download_osm_data.convert_ways_points(df_way, Data, geo_crs, distance_crs)
Convert Ways to Point Coordinates
- download_osm_data.country_list_to_geofk(country_list)
Convert the requested country list into geofk norm
- Parameters
input (str) – Any two-letter country name or aggregation of countries given in config_osm_data.py Country name duplications won’t distort the result. Examples are: [“NG”,”ZA”], downloading osm data for Nigeria and South Africa [“SNGM”], downloading data for Senegal&Gambia shape [“NG”,”ZA”,”NG”], won’t distort result.
- Returns
full_codes_list – Example [“NG”,”ZA”]
- Return type
- download_osm_data.create_country_list(input, iso_coding=True)
Create a country list for defined regions in config_osm_data.py
- Parameters
input (str) – Any two-letter country name, regional name, or continent given in config_osm_data.py Country name duplications won’t distort the result. Examples are: [“NG”,”ZA”], downloading osm data for Nigeria and South Africa [“africa”], downloading data for Africa [“NAR”], downloading data for the North African Power Pool [“TEST”], downloading data for a customized test set. [“NG”,”ZA”,”NG”], won’t distort result.
- Returns
full_codes_list – Example [“NG”,”ZA”]
- Return type
- download_osm_data.download_and_filter(feature, country_code, update=False, verify=False)
Download OpenStreetMap raw file for selected tag.
Apply pbf download and filter with esy.osmfilter selected OpenStreetMap tags or data. Examples of possible tags are listed at OpenStreetMap wiki. More information on esy.osmfilter here.
- Parameters
- Returns
substation_data (Data, Elements)
line_data (Data, Elements)
generator_data (Data, Elements) – Nested dictionary with all OpenStreetMap keys of specific component. Example of lines. See https://wiki.openstreetmap.org/wiki/Tag:power%3Dline
- download_osm_data.download_pbf(country_code, update, verify, logging=True)
Download pbf file from geofabrik for a given country code
- download_osm_data.getContinentCountry(code)
Gets continent and country name from geofabrik country code input
- download_osm_data.lonlat_lookup(df_way, Data)
Lookup refs and convert to list of longlats
- download_osm_data.output_csv_geojson(output_files, df_all_feature, columns_feature, feature, geo_crs)
Function to save the feature as csv and geojson
- download_osm_data.parallel_download_pbf(country_list, nprocesses, update=False, verify=False)
Function to download pbf data in parallel
- download_osm_data.process_data(feature_list, country_list, output_files, geo_crs, distance_crs, iso_coding=True, update=False, verify=False, nprocesses=1)
Download the features in feature_list for each country of the country_list
simplify_network
Lifts electrical transmission network to a single 380 kV voltage layer, removes dead-ends of the network, and reduces multi-hop HVDC connections to a single link.
Relevant Settings
Inputs
data/costs.csv
: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.resources/regions_onshore.geojson
: confer Rule build_bus_regionsresources/regions_offshore.geojson
: confer Rule build_bus_regionsnetworks/elec.nc
: confer Rule add_electricity
Outputs
resources/regions_onshore_elec_s{simpl}.geojson
:resources/regions_offshore_elec_s{simpl}.geojson
:resources/busmap_elec_s{simpl}.csv
: Mapping of buses fromnetworks/elec.nc
tonetworks/elec_s{simpl}.nc
;networks/elec_s{simpl}.nc
:
Description
The rule simplify_network
does up to four things:
Create an equivalent transmission network in which all voltage levels are mapped to the 380 kV level by the function
simplify_network(...)
.DC only sub-networks that are connected at only two buses to the AC network are reduced to a single representative link in the function
simplify_links(...)
. The components attached to buses in between are moved to the nearest endpoint. The grid connection cost of offshore wind generators are added to the captial costs of the generator.Stub lines and links, i.e. dead-ends of the network, are sequentially removed from the network in the function
remove_stubs(...)
. Components are moved along.Optionally, if an integer were provided for the wildcard
{simpl}
(e.g.networks/elec_s500.nc
), the network is clustered to this number of clusters with the routines from thecluster_network
rule with the functioncluster_network.cluster(...)
. This step is usually skipped!
- simplify_network.busmap_by_stubs(network, matching_attrs=None)
Create a busmap by reducing stubs and stubby trees (i.e. sequentially reducing dead-ends).
- Parameters
network (pypsa.Network) –
matching_attrs (None|[str]) – bus attributes clusters have to agree on
- Returns
busmap – Mapping of network.buses to k-means clusters (indexed by non-negative integers).
- Return type
pandas.Series
- simplify_network.simplify_network_to_380(n, linetype)
Simplify network to v_nom == 380