API Reference (automated)

_helpers

_helpers.configure_logging(snakemake, skip_handlers=False)

Configure the basic behaviour for the logging module.

Note: Must only be called once from the __main__ section of a script.

The setup includes printing log messages to STDERR and to a log file defined by either (in priority order): snakemake.log.python, snakemake.log[0] or “logs/{rulename}.log”. Additional keywords from logging.basicConfig are accepted via the snakemake configuration file under snakemake.config.logging.

Parameters

  • snakemake (snakemake object) – Your snakemake object containing a snakemake.config and snakemake.log.

  • skip_handlers (True | False (default)) – Do (not) skip the default handlers created for redirecting output to STDERR and file.

_helpers.country_name_2_two_digits(country_name)

Convert full country name to 2-digit country code

Parameters

country_name (str) – country name

Returns

two_code_country – 2-digit country name

Return type

str

_helpers.getContinent(code)

Returns continent names that contains list of iso-code countries

Parameters

code (str) – List of two letter country ISO codes

Returns

continent_list – List of continent names

Return type

str

Example

from helpers import getContinent code = [“DE”, “GB”, “NG”, “ZA”] getContinent(code) >>> [“africa”, “europe”]

_helpers.get_aggregation_strategies(aggregation_strategies)

default aggregation strategies that cannot be defined in .yaml format must be specified within the function, otherwise (when defaults are passed in the function’s definition) they get lost when custom values are specified in the config.

_helpers.get_country(target, **keys)

Function to convert country codes using pycountry

Parameters

  • target (str) –

    Desired type of country code. Examples:

    • ’alpha_3’ for 3-digit

    • ’alpha_2’ for 2-digit

    • ’name’ for full country name

  • keys (dict) –

    Specification of the country name and reference system. Examples:

    • alpha_3=”ZAF” for 3-digit

    • alpha_2=”ZA” for 2-digit

    • name=”South Africa” for full country name

Returns

  • country code as requested in keys or np.nan, when country code is not recognized

  • Example of usage

  • ——-

  • - Convert 2-digit code to 3-digit codes (get_country(‘alpha_3’, alpha_2=”ZA”))

  • - Convert 3-digit code to 2-digit codes (get_country(‘alpha_2’, alpha_3=”ZAF”))

  • - Convert 2-digit code to full name (get_country(‘name’, alpha_2=”ZA”))

_helpers.load_network(import_name=None, custom_components=None)

Helper for importing a pypsa.Network with additional custom components.

Parameters

  • import_name (str) – As in pypsa.Network(import_name)

  • custom_components (dict) –

    Dictionary listing custom components. For using snakemake.config["override_components"] in config.yaml define:

    override_components:
    ShadowPrice:
        component: ["shadow_prices","Shadow price for a global constraint.",np.nan]
        attributes:
        name: ["string","n/a","n/a","Unique name","Input (required)"]
        value: ["float","n/a",0.,"shadow value","Output"]

Return type

pypsa.Network

_helpers.mock_snakemake(rulename, **wildcards)

This function is expected to be executed from the “scripts”-directory of ” the snakemake project. It returns a snakemake.script.Snakemake object, based on the Snakefile.

If a rule has wildcards, you have to specify them in **wildcards.

Parameters

  • rulename (str) – name of the rule for which the snakemake object should be generated

  • **wildcards – keyword arguments fixing the wildcards. Only necessary if wildcards are needed.

_helpers.progress_retrieve(url, file, data=None, disable_progress=False, roundto=1.0)

Function to download data from a url with a progress bar progress in retrieving data

Parameters

  • url (str) – Url to download data from

  • file (str) – File where to save the output

  • data (dict) – Data for the request (default None), when not none Post method is used

  • disable_progress (bool) – When true, no progress bar is shown

  • roundto (float) – (default 0) Precision used to report the progress e.g. 0.1 stands for 88.1, 10 stands for 90, 80

_helpers.read_csv_nafix(file, **kwargs)

Function to open a csv as pandas file and standardize the na value

_helpers.sets_path_to_root(root_directory_name)

Search and sets path to the given root directory (root/path/file).

Parameters

  • root_directory_name (str) – Name of the root directory.

  • n (int) – Number of folders the function will check upwards/root directed.

_helpers.three_2_two_digits_country(three_code_country)

Convert 3-digit to 2-digit country code:

Parameters

three_code_country (str) – 3-digit country name

Returns

two_code_country – 2-digit country name

Return type

str

_helpers.two_2_three_digits_country(two_code_country)

Convert 2-digit to 3-digit country code:

Parameters

two_code_country (str) – 2-digit country name

Returns

three_code_country – 3-digit country name

Return type

str

_helpers.two_digits_2_name_country(two_code_country, nocomma=False, remove_start_words=[])

Convert 2-digit country code to full name country:

Parameters

  • two_code_country (str) – 2-digit country name

  • nocomma (bool (optional, default False)) – When true, country names with comma are extended to remove the comma. Example CD -> Congo, The Democratic Republic of -> The Democratic Republic of Congo

  • remove_start_words (list (optional, default empty)) – When a sentence starts with any of the provided words, the beginning is removed. e.g. The Democratic Republic of Congo -> Democratic Republic of Congo (remove_start_words=[“The”])

Returns

full_name – full country name

Return type

str

add_electricity

Adds electrical generators, load and existing hydro storage units to a base network.

Relevant Settings

costs:
    year:
    USD2013_to_EUR2013:
    dicountrate:
    emission_prices:

electricity:
    max_hours:
    marginal_cost:
    capital_cost:
    conventional_carriers:
    co2limit:
    extendable_carriers:
    include_renewable_capacities_from_OPSD:
    estimate_renewable_capacities_from_capacity_stats:

load:
    scale:
    ssp:
    weather_year:
    prediction_year:
    region_load:


renewable:
    hydro:
        carriers:
        hydro_max_hours:
        hydro_capital_cost:

lines:
    length_factor:

See also

Documentation of the configuration file config.yaml at costs, electricity, load, renewable, lines

Inputs

  • data/costs.csv: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.

  • data/bundle/hydro_capacities.csv: Hydropower plant store/discharge power capacities, energy storage capacity, and average hourly inflow by country. Not currently used!

    ../img/hydrocapacities.png

  • data/geth2015_hydro_capacities.csv: alternative to capacities above; not currently used!

  • resources/ssp2-2.6/2030/era5_2013/Africa.nc Hourly country load profiles produced by GEGIS

  • resources/regions_onshore.geojson: confer Rule build_bus_regions

  • resources/gadm_shapes.geojson: confer Rule build_shapes

  • resources/powerplants.csv: confer Rule build_powerplants

  • resources/profile_{}.nc: all technologies in config["renewables"].keys(), confer Rule build_renewable_profiles.

  • networks/base.nc: confer Rule base_network

Outputs

  • networks/elec.nc:

    ../img/elec.png

Description

The rule add_electricity ties all the different data inputs from the preceding rules together into a detailed PyPSA network that is stored in networks/elec.nc. It includes:

  • today’s transmission topology and transfer capacities (in future, optionally including lines which are under construction according to the config settings lines: under_construction and links: under_construction),

  • today’s thermal and hydro power generation capacities (for the technologies listed in the config setting electricity: conventional_carriers), and

  • today’s load time-series (upsampled in a top-down approach according to population and gross domestic product)

It further adds extendable generators with zero capacity for

  • photovoltaic, onshore and AC- as well as DC-connected offshore wind installations with today’s locational, hourly wind and solar capacity factors (but no current capacities),

  • additional open- and combined-cycle gas turbines (if OCGT and/or CCGT is listed in the config setting electricity: extendable_carriers)

add_electricity.attach_load(n, load_paths, regions, admin_shapes, countries, scale)

Add load to the network and distributes them according GDP and population.

Parameters

  • n (pypsa network) –

  • regions (.geojson) – Contains bus_id of low voltage substations and bus region shapes (voronoi cells)

  • load_paths (paths of the load files) –

  • admin_shapes (.geojson) – contains subregional gdp, population and shape data

  • countries (list) – List of countries that is config input

  • scale (float) – The scale factor is multiplied with the load (1.3 = 30% more load)

Returns

n – Now attached with load time series

Return type

pypsa network

add_electricity.calculate_annuity(n, r)

Calculate the annuity factor for an asset with lifetime n years and discount rate of r, e.g. annuity(20, 0.05) * 20 = 1.6

add_electricity.get_load_paths_gegis(ssp_parentfolder, config)

Creates load paths for the GEGIS outputs

The paths are created automatically according to included country, weather year, prediction year and ssp scenario

Example

[“/data/ssp2-2.6/2030/era5_2013/Africa.nc”, “/data/ssp2-2.6/2030/era5_2013/Africa.nc”]

add_electricity.load_costs(tech_costs, config, elec_config, Nyears=1)

set all asset costs and other parameters

base_network

Creates the network topology from a OpenStreetMap

Relevant Settings

snapshots:

countries:

electricity:
    voltages:

lines:
    types:
    s_max_pu:
    under_construction:

links:
    p_max_pu:
    under_construction:
    include_tyndp:

transformers:
    x:
    s_nom:
    type:

See also

Documentation of the configuration file config.yaml at snapshots, Top-level configuration, electricity, load, lines, links, transformers

Inputs

Outputs

  • networks/base.nc

    ../img/base.png

Description

build_bus_regions

Creates Voronoi shapes for each bus representing both onshore and offshore regions.

Relevant Settings

countries:

See also

Documentation of the configuration file config.yaml at Top-level configuration

Inputs

Outputs

  • resources/regions_onshore.geojson:

    ../img/regions_onshore.png

  • resources/regions_offshore.geojson:

    ../img/regions_offshore.png

Description

build_bus_regions.custom_voronoi_partition_pts(points, outline, add_bounds_shape=True, multiplier=5)

Compute the polygons of a voronoi partition of points within the polygon outline

build_bus_regions.points
Type

Nx2 - ndarray[dtype=float]

build_bus_regions.outline
Type

Polygon

build_bus_regions.no_multipolygons

If true, replace each MultiPolygon by its largest component

Type

bool (default: False)

Returns

polygons

Return type

N - ndarray[dtype=Polygon|MultiPolygon]

build_cutout

Create cutouts with atlite.

For this rule to work you must have

See also

For details on the weather data read the atlite documentation. If you need help specifically for creating cutouts the corresponding section in the atlite documentation should be helpful.

Relevant Settings

atlite:
    nprocesses:
    cutouts:
        {cutout}:

See also

Documentation of the configuration file config.yaml at atlite

Inputs

None

Outputs

  • cutouts/{cutout}: weather data from either the ERA5 reanalysis weather dataset or SARAH-2 satellite-based historic weather data with the following structure:

ERA5 cutout:

Field

Dimensions

Unit

Description

pressure

time, y, x

Pa

Surface pressure

temperature

time, y, x

K

Air temperature 2 meters above the surface.

soil temperature

time, y, x

K

Soil temperature between 1 meters and 3 meters depth (layer 4).

influx_toa

time, y, x

Wm**-2

Top of Earth’s atmosphere TOA incident solar radiation

influx_direct

time, y, x

Wm**-2

Total sky direct solar radiation at surface

runoff

time, y, x

m

Runoff (volume per area)

roughness

y, x

m

Forecast surface roughness (roughness length)

height

y, x

m

Surface elevation above sea level

albedo

time, y, x

–

Albedo measure of diffuse reflection of solar radiation. Calculated from relation between surface solar radiation downwards (Jm**-2) and surface net solar radiation (Jm**-2). Takes values between 0 and 1.

influx_diffuse

time, y, x

Wm**-2

Diffuse solar radiation at surface. Surface solar radiation downwards minus direct solar radiation.

wnd100m

time, y, x

ms**-1

Wind speeds at 100 meters (regardless of direction)
../img/era5.png

A SARAH-2 cutout can be used to amend the fields temperature, influx_toa, influx_direct, albedo, influx_diffuse of ERA5 using satellite-based radiation observations.

../img/sarah.png

Description

build_natura_raster

Converts vectordata or known as shapefiles (i.e. used for geopandas/shapely) to our cutout rasters. The Protected Planet Data on protected areas is aggregated to all cutout regions.

Relevant Settings

renewable:
    {technology}:
        cutout:

See also

Documentation of the configuration file config.yaml at renewable

Inputs

  • data/raw/protected_areas/WDPA_WDOECM_Aug2021_Public_AF_shp-points.shp: WDPA World Database for Protected Areas.

    ../img/natura.png

Outputs

  • resources/natura.tiff: Rasterized version of Natura 2000 natural protection areas to reduce computation times.

    ../img/natura.png

Description

To operate the script you need all input files. The Snakefile describes what goes in and out. Make sure you didn’t skip one of these. Maybe not so obvious is the cutout input. An example is this africa-2013-era5.nc

Steps to retrieve the protected area data (as apparently no API is given for the WDPA data):

    1. Download the WPDA Dataset: World Database on Protected Areas. UNEP-WCMC and IUCN (2021), Protected Planet: The World Database on Protected Areas (WDPA) and World Database on Other Effective Area-based Conservation Measures (WD-OECM) [Online], August 2021, Cambridge, UK: UNEP-WCMC and IUCN. Available at: www.protectedplanet.net.

    1. Unzipp and rename the folder containing the .shp file to protected_areas

    1. Important! Don’t delete the other files which come with the .shp file. They are required to build the shape.

    1. Move the file in such a way that the above path is given

    1. Activate the environment of environment-max.yaml

    1. Ready to run the script

Tip: The output file natura.tiff contains now the 100x100m rasters of protective areas. This operation can make the filesize of that TIFF quite large and leads to problems when trying to open. QGIS, an open source tool helps exploring the file.

build_natura_raster.get_fileshapes(list_paths, accepted_formats=('.shp',))

Function to parse the list of paths to include shapes included in folders, if any

build_natura_raster.unify_protected_shape_areas(inputs, area_crs, out_logging)

Iterates thorugh all snakemake rule inputs and unifies shapefiles (.shp) only.

The input is given in the Snakefile and shapefiles are given by .shp

Returns

unified_shape

Return type

GeoDataFrame with a unified “multishape”

build_renewable_profiles

Calculates for each network node the (i) installable capacity (based on land-use), (ii) the available generation time series (based on weather data), and (iii) the average distance from the node for onshore wind, AC-connected offshore wind, DC-connected offshore wind and solar PV generators. For hydro generators, it calculates the expected inflows. In addition for offshore wind it calculates the fraction of the grid connection which is under water.

Relevant settings

snapshots:

atlite:
    nprocesses:

renewable:
    {technology}:
        cutout:
        copernicus:
            grid_codes:
            distance:
            distance_grid_codes:
        natura:
        max_depth:
        max_shore_distance:
        min_shore_distance:
        capacity_per_sqkm:
        correction_factor:
        potential:
        min_p_max_pu:
        clip_p_max_pu:
        resource:
        clip_min_inflow:

See also

Documentation of the configuration file config.yaml at snapshots, atlite, renewable

Inputs

Outputs

  • resources/profile_{technology}.nc, except hydro technology, with the following structure

    Field

    Dimensions

    Description

    profile

    bus, time

    the per unit hourly availability factors for each node

    weight

    bus

    sum of the layout weighting for each node

    p_nom_max

    bus

    maximal installable capacity at the node (in MW)

    potential

    y, x

    layout of generator units at cutout grid cells inside the Voronoi cell (maximal installable capacity at each grid cell multiplied by capacity factor)

    average_distance

    bus

    average distance of units in the Voronoi cell to the grid node (in km)

    underwater_fraction

    bus

    fraction of the average connection distance which is under water (only for offshore)
  • resources/profile_hydro.nc for the hydro technology

    Field

    Dimensions

    Description

    inflow

    plant, time

    Inflow to the state of charge (in MW), e.g. due to river inflow in hydro reservoir.

    • profile

    ../img/profile_ts.png

    • p_nom_max

    ../img/p_nom_max_hist.png

    • potential

    ../img/potential_heatmap.png

    • average_distance

    ../img/distance_hist.png

    • underwater_fraction

    ../img/underwater_hist.png

Description

This script leverages on atlite function to derivate hourly time series for an entire year for solar, wind (onshore and offshore), and hydro data.

This script functions at two main spatial resolutions: the resolution of the network nodes and their Voronoi cells, and the resolution of the cutout grid cells for the weather data. Typically the weather data grid is finer than the network nodes, so we have to work out the distribution of generators across the grid cells within each Voronoi cell. This is done by taking account of a combination of the available land at each grid cell and the capacity factor there.

This uses the Copernicus land use data, Natura2000 nature reserves and GEBCO bathymetry data.

../img/eligibility.png

To compute the layout of generators in each node’s Voronoi cell, the installable potential in each grid cell is multiplied with the capacity factor at each grid cell. This is done since we assume more generators are installed at cells with a higher capacity factor.

../img/offwinddc-gridcell.png ../img/offwindac-gridcell.png ../img/onwind-gridcell.png ../img/solar-gridcell.png

This layout is then used to compute the generation availability time series from the weather data cutout from atlite.

Two methods are available to compute the maximal installable potential for the node (p_nom_max): simple and conservative:

  • simple adds up the installable potentials of the individual grid cells. If the model comes close to this limit, then the time series may slightly overestimate production since it is assumed the geographical distribution is proportional to capacity factor.

  • conservative assertains the nodal limit by increasing capacities proportional to the layout until the limit of an individual grid cell is reached.

build_shapes

build_shapes.add_gdp_data(df_gadm, year=2020, update=False, out_logging=False, name_file_nc='GDP_PPP_1990_2015_5arcmin_v2.nc', nprocesses=2, disable_progressbar=False)

Function to add gdp data to arbitrary number of shapes in a country

df_gadm: Geodataframe with one Multipolygon per row

  • Essential column [“country”, “geometry”]

  • Non-essential column [“GADM_ID”]

df_gadm: Geodataframe with one Multipolygon per row

  • Same columns as input

  • Includes a new column [“gdp”]

build_shapes.add_population_data(df_gadm, country_codes, worldpop_method, year=2020, update=False, out_logging=False, nprocesses=2, disable_progressbar=False)

Function to add population data to arbitrary number of shapes in a country

df_gadm: Geodataframe with one Multipolygon per row

  • Essential column [“country”, “geometry”]

  • Non-essential column [“GADM_ID”]

df_gadm: Geodataframe with one Multipolygon per row

  • Same columns as input

  • Includes a new column [“pop”]

build_shapes.convert_GDP(name_file_nc, year=2015, out_logging=False)

Function to convert the nc database of the GDP to tif, based on the work at https://doi.org/10.1038/sdata.2018.4. The dataset shall be downloaded independently by the user (see guide) or toghether with pypsa-africa package.

build_shapes.countries(countries, geo_crs, update=False, out_logging=False)

Create country shapes

build_shapes.download_GADM(country_code, update=False, out_logging=False)

Download gpkg file from GADM for a given country code

Parameters

  • country_code (str) – Two letter country codes of the downloaded files

  • update (bool) – Update = true, forces re-download of files

Return type

gpkg file per country

build_shapes.download_WorldPop(country_code, worldpop_method, year=2020, update=False, out_logging=False, size_min=300)
Download Worldpop using either the standard method or the API method.
worldpop_method: str

worldpop_method = “api” will use the API method to access the WorldPop 100mx100m dataset. worldpop_method = “standard” will use the standard method to access the WorldPop 1KMx1KM dataset.

country_codestr

Two letter country codes of the downloaded files. Files downloaded from https://data.worldpop.org/ datasets WorldPop UN adjusted

yearint

Year of the data to download

updatebool

Update = true, forces re-download of files

size_minint

Minimum size of each file to download

build_shapes.download_WorldPop_API(country_code, year=2020, update=False, out_logging=False, size_min=300)

Download tiff file for each country code using the api method from worldpop API with 100mx100m resolution.

Parameters

  • country_code (str) – Two letter country codes of the downloaded files. Files downloaded from https://data.worldpop.org/ datasets WorldPop UN adjusted

  • year (int) – Year of the data to download

  • update (bool) – Update = true, forces re-download of files

  • size_min (int) – Minimum size of each file to download

Returns

  • WorldPop_inputfile (str) – Path of the file

  • WorldPop_filename (str) – Name of the file

build_shapes.download_WorldPop_standard(country_code, year=2020, update=False, out_logging=False, size_min=300)

Download tiff file for each country code using the standard method from worldpop datastore with 1kmx1km resolution.

Parameters

  • country_code (str) – Two letter country codes of the downloaded files. Files downloaded from https://data.worldpop.org/ datasets WorldPop UN adjusted

  • year (int) – Year of the data to download

  • update (bool) – Update = true, forces re-download of files

  • size_min (int) – Minimum size of each file to download

Returns

  • WorldPop_inputfile (str) – Path of the file

  • WorldPop_filename (str) – Name of the file

build_shapes.eez(countries, geo_crs, country_shapes, EEZ_gpkg, out_logging=False, distance=0.01)

Creates offshore shapes by - buffer smooth countryshape (=offset country shape) - and differ that with the offshore shape Leads to for instance a 100m non-build coastline

build_shapes.generalized_mask(src, geom, **kwargs)

Generalize mask function to account for Polygon and MultiPolygon

build_shapes.get_GADM_layer(country_list, layer_id, geo_crs, update=False, outlogging=False)

Function to retrive a specific layer id of a geopackage for a selection of countries

Parameters

  • country_list (str) – List of the countries

  • layer_id (int) – Layer to consider in the format GID_{layer_id}. When the requested layer_id is greater than the last available layer, then the last layer is selected. When a negative value is requested, then, the last layer is requested

build_shapes.load_EEZ(countries_codes, geo_crs, EEZ_gpkg='./data/eez/eez_v11.gpkg')

Function to load the database of the Exclusive Economic Zones. The dataset shall be downloaded independently by the user (see guide) or together with pypsa-africa package.

build_shapes.load_GDP(countries_codes, year=2015, update=False, out_logging=False, name_file_nc='GDP_PPP_1990_2015_5arcmin_v2.nc')

Function to load the database of the GDP, based on the work at https://doi.org/10.1038/sdata.2018.4. The dataset shall be downloaded independently by the user (see guide) or toghether with pypsa-africa package.

cluster_network

Creates networks clustered to {cluster} number of zones with aggregated buses, generators and transmission corridors.

Relevant Settings

clustering:
    aggregation_strategies:

focus_weights:

solving:
    solver:
        name:

lines:
    length_factor:

See also

Documentation of the configuration file config.yaml at Top-level configuration, renewable, solving, lines

Inputs

Outputs

  • resources/regions_onshore_elec_s{simpl}_{clusters}.geojson:

    ../img/regions_onshore_elec_s_X.png

  • resources/regions_offshore_elec_s{simpl}_{clusters}.geojson:

    ../img/regions_offshore_elec_s_X.png

  • resources/busmap_elec_s{simpl}_{clusters}.csv: Mapping of buses from networks/elec_s{simpl}.nc to networks/elec_s{simpl}_{clusters}.nc;

  • resources/linemap_elec_s{simpl}_{clusters}.csv: Mapping of lines from networks/elec_s{simpl}.nc to networks/elec_s{simpl}_{clusters}.nc;

  • networks/elec_s{simpl}_{clusters}.nc:

    ../img/elec_s_X.png

Description

Note

Why is clustering used both in simplify_network and cluster_network ?

Consider for example a network networks/elec_s100_50.nc in which simplify_network clusters the network to 100 buses and in a second step cluster_network` reduces it down to 50 buses.

In preliminary tests, it turns out, that the principal effect of changing spatial resolution is actually only partially due to the transmission network. It is more important to differentiate between wind generators with higher capacity factors from those with lower capacity factors, i.e. to have a higher spatial resolution in the renewable generation than in the number of buses.

The two-step clustering allows to study this effect by looking at networks like networks/elec_s100_50m.nc. Note the additional m in the {cluster} wildcard. So in the example network there are still up to 100 different wind generators.

In combination these two features allow you to study the spatial resolution of the transmission network separately from the spatial resolution of renewable generators.

Is it possible to run the model without the simplify_network rule?

No, the network clustering methods in the PyPSA module pypsa.networkclustering do not work reliably with multiple voltage levels and transformers.

Tip

The rule cluster_all_networks runs for all scenario s in the configuration file the rule cluster_network.

Exemplary unsolved network clustered to 512 nodes:

../img/elec_s_512.png

Exemplary unsolved network clustered to 256 nodes:

../img/elec_s_256.png

Exemplary unsolved network clustered to 128 nodes:

../img/elec_s_128.png

Exemplary unsolved network clustered to 37 nodes:

../img/elec_s_37.png
cluster_network.distribute_clusters(n, n_clusters, focus_weights=None, solver_name=None)

Determine the number of clusters per country

build_osm_network

build_osm_network.connect_stations_same_station_id(lines, buses)

Function to create fake links between substations with the same substation_id

build_osm_network.fix_overpassing_lines(lines, buses, distance_crs, tol=1)

Function to avoid buses overpassing lines with no connection when the bus is within a given tolerance from the line

Parameters

  • lines (GeoDataFrame) – Geodataframe of lines

  • buses (GeoDataFrame) – Geodataframe of substations

  • tol (float) – Tolerance in meters of the distance between the substation and the line below which the line will be splitted

build_osm_network.get_ac_frequency(df, fr_col='tag_frequency')

# Function to define a default frequency value. Attempts to find the most usual non-zero frequency across the dataframe; 50 Hz is assumed as a back-up value

build_osm_network.get_converters(buses, lines)

Function to create fake converter lines that connect buses of the same station_id of different polarities

build_osm_network.get_transformers(buses, lines)

Function to create fake transformer lines that connect buses of the same station_id at different voltage

build_osm_network.merge_stations_lines_by_station_id_and_voltage(lines, buses, geo_crs, distance_crs, tol=2000)

Function to merge close stations and adapt the line datasets to adhere to the merged dataset

build_osm_network.merge_stations_same_station_id(buses, delta_lon=0.001, delta_lat=0.001, precision=4)

Function to merge buses with same voltage and station_id This function iterates over all substation ids and creates a bus_id for every substation and voltage level. Therefore, a substation with multiple voltage levels is represented with different buses, one per voltage level

build_osm_network.set_lines_ids(lines, buses, distance_crs)

Function to set line buses ids to the closest bus in the list

build_osm_network.set_lv_substations(buses)

Function to set what nodes are lv, thereby setting substation_lv The current methodology is to set lv nodes to buses where multiple voltage level are found, hence when the station_id is duplicated

build_osm_network.set_substations_ids(buses, distance_crs, tol=2000)

Function to set substations ids to buses, accounting for location tolerance

The algorithm is as follows:

  1. initialize all substation ids to -1

  2. if the current substation has been already visited [substation_id < 0], then skip the calculation

  3. otherwise:

    1. identify the substations within the specified tolerance (tol)

    2. when all the substations in tolerance have substation_id < 0, then specify a new substation_id

    3. otherwise, if one of the substation in tolerance has a substation_id >= 0, then set that substation_id to all the others; in case of multiple substations with substation_ids >= 0, the first value is picked for all

clean_osm_data

config_osm_data

download_osm_data

OSM extraction script.

download_osm_data.convert_filtered_data_to_dfs(country_code, feature_data, feature)

Convert Filtered Data, Elements to Pandas Dataframes

download_osm_data.convert_iso_to_geofk(iso_code, iso_coding=True, convert_dict={'AE': 'GCC', 'BH': 'GCC', 'BN': 'MY-SG-BN', 'EH': 'MA', 'GM': 'SNGM', 'HK': 'CN', 'IL': 'IL-PL', 'KW': 'GCC', 'MO': 'CN', 'MY': 'MY-SG-BN', 'OM': 'GCC', 'PS': 'IL-PL', 'QA': 'GCC', 'SA': 'GCC', 'SG': 'MY-SG-BN', 'SM': 'IT', 'SN': 'SNGM'})

Function to convert the iso code name of a country into the corresponding geofabrik In Geofabrik, some countries are aggregated, thus if a single country is requested, then all the agglomeration shall be downloaded For example, Senegal (SN) and Gambia (GM) cannot be found alone in geofabrik, but they can be downloaded as a whole SNGM

The conversion directory, initialized to iso_to_geofk_dict is used to perform such conversion When a two-letter code country is found in convert_dict, and iso_coding is enabled, then that two-letter code is converted into the corresponding value of the dictionary

Parameters

  • iso_code (str) – Two-code country code to be converted

  • iso_coding (bool) – When true, the iso to geofk is performed

  • convert_dict (dict) – Dictionary used to apply the conversion iso to geofk The keys correspond to the countries iso codes that need a different region to be downloaded

download_osm_data.convert_pd_to_gdf_lines(df_way, geo_crs, simplified=False)

Convert Lines Pandas Dataframe to GeoPandas Dataframe

download_osm_data.convert_pd_to_gdf_nodes(df_way, geo_crs)

Convert Points Pandas Dataframe to GeoPandas Dataframe

download_osm_data.convert_ways_lines(df_way, Data, geo_crs, distance_crs)

Convert Ways to Line Coordinates

download_osm_data.convert_ways_points(df_way, Data, geo_crs, distance_crs)

Convert Ways to Point Coordinates

download_osm_data.country_list_to_geofk(country_list)

Convert the requested country list into geofk norm

Parameters

input (str) – Any two-letter country name or aggregation of countries given in config_osm_data.py Country name duplications won’t distort the result. Examples are: [“NG”,”ZA”], downloading osm data for Nigeria and South Africa [“SNGM”], downloading data for Senegal&Gambia shape [“NG”,”ZA”,”NG”], won’t distort result.

Returns

full_codes_list – Example [“NG”,”ZA”]

Return type

list

download_osm_data.create_country_list(input, iso_coding=True)

Create a country list for defined regions in config_osm_data.py

Parameters

input (str) – Any two-letter country name, regional name, or continent given in config_osm_data.py Country name duplications won’t distort the result. Examples are: [“NG”,”ZA”], downloading osm data for Nigeria and South Africa [“africa”], downloading data for Africa [“NAR”], downloading data for the North African Power Pool [“TEST”], downloading data for a customized test set. [“NG”,”ZA”,”NG”], won’t distort result.

Returns

full_codes_list – Example [“NG”,”ZA”]

Return type

list

download_osm_data.download_and_filter(feature, country_code, update=False, verify=False)

Download OpenStreetMap raw file for selected tag.

Apply pbf download and filter with esy.osmfilter selected OpenStreetMap tags or data. Examples of possible tags are listed at OpenStreetMap wiki. More information on esy.osmfilter here.

Parameters

  • country_code (str) – Three letter country codes of the downloaded files

  • update (bool) – Name of the network component Update = true, forces re-download of files Update = false, uses existing or previously downloaded files to safe time

Returns

download_osm_data.download_pbf(country_code, update, verify, logging=True)

Download pbf file from geofabrik for a given country code

Parameters

  • country_code (str) – Three letter country codes of the downloaded files

  • update (bool) – Name of the network component Update = true, forces re-download of files

Return type

Pbf file per country

download_osm_data.getContinentCountry(code)

Gets continent and country name from geofabrik country code input

download_osm_data.lonlat_lookup(df_way, Data)

Lookup refs and convert to list of longlats

download_osm_data.output_csv_geojson(output_files, df_all_feature, columns_feature, feature, geo_crs)

Function to save the feature as csv and geojson

download_osm_data.parallel_download_pbf(country_list, nprocesses, update=False, verify=False)

Function to download pbf data in parallel

Parameters

  • country_list (str) – List of geofabrik country codes to download

  • nprocesses (int) – Number of parallel processes

  • update (bool) – If true, existing pbf files are updated. Default: False

  • verify (bool) – If true, checks the md5 of the file. Default: False

download_osm_data.process_data(feature_list, country_list, output_files, geo_crs, distance_crs, iso_coding=True, update=False, verify=False, nprocesses=1)

Download the features in feature_list for each country of the country_list

simplify_network

Lifts electrical transmission network to a single 380 kV voltage layer, removes dead-ends of the network, and reduces multi-hop HVDC connections to a single link.

Relevant Settings

See also

Documentation of the configuration file config.yaml at costs, electricity, renewable, lines, links, solving

Inputs

  • data/costs.csv: The database of cost assumptions for all included technologies for specific years from various sources; e.g. discount rate, lifetime, investment (CAPEX), fixed operation and maintenance (FOM), variable operation and maintenance (VOM), fuel costs, efficiency, carbon-dioxide intensity.

  • resources/regions_onshore.geojson: confer Rule build_bus_regions

  • resources/regions_offshore.geojson: confer Rule build_bus_regions

  • networks/elec.nc: confer Rule add_electricity

Outputs

  • resources/regions_onshore_elec_s{simpl}.geojson:

    ../img/regions_onshore_elec_s.png

  • resources/regions_offshore_elec_s{simpl}.geojson:

    ../img/regions_offshore_elec_s.png

  • resources/busmap_elec_s{simpl}.csv: Mapping of buses from networks/elec.nc to networks/elec_s{simpl}.nc;

  • networks/elec_s{simpl}.nc:

    ../img/elec_s.png

Description

The rule simplify_network does up to four things:

  1. Create an equivalent transmission network in which all voltage levels are mapped to the 380 kV level by the function simplify_network(...).

  2. DC only sub-networks that are connected at only two buses to the AC network are reduced to a single representative link in the function simplify_links(...). The components attached to buses in between are moved to the nearest endpoint. The grid connection cost of offshore wind generators are added to the captial costs of the generator.

  3. Stub lines and links, i.e. dead-ends of the network, are sequentially removed from the network in the function remove_stubs(...). Components are moved along.

  4. Optionally, if an integer were provided for the wildcard {simpl} (e.g. networks/elec_s500.nc), the network is clustered to this number of clusters with the routines from the cluster_network rule with the function cluster_network.cluster(...). This step is usually skipped!

simplify_network.busmap_by_stubs(network, matching_attrs=None)

Create a busmap by reducing stubs and stubby trees (i.e. sequentially reducing dead-ends).

Parameters

  • network (pypsa.Network) –

  • matching_attrs (None|[str]) – bus attributes clusters have to agree on

Returns

busmap – Mapping of network.buses to k-means clusters (indexed by non-negative integers).

Return type

pandas.Series

simplify_network.simplify_network_to_380(n, linetype)

Simplify network to v_nom == 380