Enhancing Tags in an OSM File¶

How to modify selected objects in an OSM file.

Task¶

Localise the OSM file for the French language: when a name:fr tag is available, replace the name tag with it and save the original name in name:local.

Quick solution¶

In [1]:

import osmium

import osmium

In [2]:

with osmium.SimpleWriter('../data/out/renamed.pbf', overwrite=True) as writer:
    fp = osmium.FileProcessor('../data/liechtenstein.osm.pbf')\
               .with_filter(osmium.filter.KeyFilter('name:fr'))\
               .handler_for_filtered(writer)

    for obj in fp:
        # start with a set of tags without name:fr
        tags = {k: v for k, v in obj.tags if k != 'name:fr'}
        # replace the name tag with the French version
        tags['name'] = obj.tags['name:fr']
        # Save the original if it exists.
        if 'name' in obj.tags:
            tags['name:local'] = obj.tags['name']
        # Write back the object with the modified tags
        writer.add(obj.replace(tags=tags))

with osmium.SimpleWriter('../data/out/renamed.pbf', overwrite=True) as writer: fp = osmium.FileProcessor('../data/liechtenstein.osm.pbf')\ .with_filter(osmium.filter.KeyFilter('name:fr'))\ .handler_for_filtered(writer) for obj in fp: # start with a set of tags without name:fr tags = {k: v for k, v in obj.tags if k != 'name:fr'} # replace the name tag with the French version tags['name'] = obj.tags['name:fr'] # Save the original if it exists. if 'name' in obj.tags: tags['name:local'] = obj.tags['name'] # Write back the object with the modified tags writer.add(obj.replace(tags=tags))

Background¶

To change selected tags in a file, it is necessary to read the file object by object, make changes as necessary and write back the data into a new file. This could be done with a simple FileProcessor (for reading the input file) that is combined with a SimpleWriter (for writing the output file):

In [3]:

with osmium.SimpleWriter('../data/out/ele.osm.opl', overwrite=True) as writer:
    for obj in osmium.FileProcessor('../data/liechtenstein.osm.pbf'):
        if 'name:fr' in obj.tags:
            tags = {k: v for k, v in obj.tags if k != 'name:fr'}
            # ... do more stuff here
            writer.add(obj.replace(tags=tags))
        else:
            writer.add(obj)

with osmium.SimpleWriter('../data/out/ele.osm.opl', overwrite=True) as writer: for obj in osmium.FileProcessor('../data/liechtenstein.osm.pbf'): if 'name:fr' in obj.tags: tags = {k: v for k, v in obj.tags if k != 'name:fr'} # ... do more stuff here writer.add(obj.replace(tags=tags)) else: writer.add(obj)

If you run this code snippet on a large OSM file, it will take a very long time to execute. Even though we only want to change a handful of objects (all objects that have a name:fr tag), the FileProcessor needs to present every single object to the Python code in the loop because every single objects needs to be written in the output file. We need a way to tell the FileProcessor to directly write out all the objects that we are not inspecting in the for loop. This can be done with the handler_for_filtered() function. It allows to define a handler for all the objects, the with_filter() handlers have rejected. The SimpleWriter class can itself function as a handler. By setting it as the handler for filtered objects, they will be directly passed to the writer.

With the SimpleWriter as fallback in place, we can now create a FileProcessor that filters for objects with a name:fr tag:

In [4]:

with osmium.SimpleWriter('../data/out/buildings.opl', overwrite=True) as writer:
    fp = osmium.FileProcessor('../data/liechtenstein.osm.pbf')\
               .with_filter(osmium.filter.KeyFilter('name:fr'))\
               .handler_for_filtered(writer)

    for obj in fp:
        print(f"{obj.id} has the French name {obj.tags['name']}.")

with osmium.SimpleWriter('../data/out/buildings.opl', overwrite=True) as writer: fp = osmium.FileProcessor('../data/liechtenstein.osm.pbf')\ .with_filter(osmium.filter.KeyFilter('name:fr'))\ .handler_for_filtered(writer) for obj in fp: print(f"{obj.id} has the French name {obj.tags['name']}.")

1932181216 has the French name Vaduz.
3696525426 has the French name Liechtenstein.
9798887324 has the French name Schweizerisches Generalkonsulat.
159018431 has the French name Rhein.
424375869 has the French name Rhein.
8497 has the French name Rhein-Route.
12464 has the French name Seen-Route.
16239 has the French name Österreich.
19664 has the French name Seen-Route - Etappe 9.
27939 has the French name Cycling in Switzerland.
51701 has the French name Schweiz/Suisse/Svizzera/Svizra.
74942 has the French name Vorarlberg.
102638 has the French name Rhein-Route - Etappe 3.
102666 has the French name Österreich - Schweiz.
102877 has the French name Österreich — Liechtenstein.
123924 has the French name Rhein.
302442 has the French name Schweizer Hauptstrassen.
1155955 has the French name Liechtenstein.
1550322 has the French name Österreich — Schweiz / Suisse / Svizzera.
1665395 has the French name Via Alpina Red.
1686631 has the French name Graubünden/Grischun/Grigioni.
1687006 has the French name Sankt Gallen.
2128682 has the French name Rätikon.
2171555 has the French name EuroVelo 15 - Rheinradweg.
2668952 has the French name European Union / Union Européenne / Europäische Union.
2698607 has the French name Alps.
11342353 has the French name Appenzeller Alpen.
12579662 has the French name Via Alpina Green.
12729625 has the French name Eurozone.
13376469 has the French name Member States of the European Union / États members de l'Union européenne / Mitgliedstaaten der Europäischen Union.

If you run this piece of code, you will notice that suddenly all objects with a French name are missing from output file. This happens because once a file is presented to Python, the SimpleWriter object doesn't see it anymore. You have to explicitly call one of the 'add' functions of the SimpleWriter to write the modified object. So the full code is:

In [5]:

with osmium.SimpleWriter('../data/out/buildings.opl', overwrite=True) as writer:
    fp = osmium.FileProcessor('../data/liechtenstein.osm.pbf')\
               .with_filter(osmium.filter.KeyFilter('name:fr'))\
               .handler_for_filtered(writer)
    for obj in fp:
        tags = {k: v for k, v in obj.tags if k != 'name:fr'}
        # ... do more stuff here
        writer.add(obj.replace(tags=tags))

with osmium.SimpleWriter('../data/out/buildings.opl', overwrite=True) as writer: fp = osmium.FileProcessor('../data/liechtenstein.osm.pbf')\ .with_filter(osmium.filter.KeyFilter('name:fr'))\ .handler_for_filtered(writer) for obj in fp: tags = {k: v for k, v in obj.tags if k != 'name:fr'} # ... do more stuff here writer.add(obj.replace(tags=tags))