nuScenes Map Expansion Tutorial

This is the tutorial for the nuScenes map expansion. In particular, the NuScenesMap data class.

This tutorial will go through the description of each layers, how we retrieve and query a certain record within the map layers, render methods, and advanced data exploration

In database terms, layers are basically tables of the map database in which we assign arbitrary parts of the maps with informative labels such as traffic_light, stop_line, walkway, etc. Refer to the discussion on layers for more details.

Setup

To install the map expansion, please download the files from https://www.nuscenes.org/download and copy the files into your nuScenes map folder, e.g. /data/sets/nuscenes/maps.

Initialization

We will be working with the singapore-onenorth map. The NuScenesMap can be initialized as follows:

Visualization

Before we go into the details, let's visualize the map.

Rendering multiple layers

The NuScenesMap class makes it possible to render multiple map layers on a matplotlib figure.

Rendering the lidar basemap

New: We can render the HD lidar basemap used for localization. The basemap is a bitmap image that can be underlaid for most functions (render_centerlines, render_egoposes_on_fancy_map, render_layers, render_map_patch, render_next_roads, render_record). The same BitMap class can also be used to render the semantic prior (drivable surface + sidewalk) from the original nuScenes release. Note that in this visualization we only show the lane annotations for better visibility.

Rendering a particular record of the map layer

We can render a record, which will show its global and local view

Rendering binary map mask layers

The NuScenesMap class makes it possible to convert multiple map layers into binary mask and render on a Matplotlib figure. First let's call get_map_mask to look at the raw data of two layers:

Now we directly visualize the map mask retrieved above using render_map_mask:

We can also render the same map rotated by 45 degrees clockwise:

Rendering map layers on top of camera images

Let us take a nuScenes camera image and overlay the relevant map layers. Note that the projections are not perfect if the ground is uneven as the localization is 2d.

Rendering ego poses on the map

We can also plot the ego poses onto the map. This requires us to load up the NuScenes class, which can take some time.

We also provide functions for navigation around the road network. For this purpose, the road layers lane, road_block and road_segment are especially useful (see definitions below). The get_next_roads(x, y) function looks at the road layer at a particular point. It then retrieves the next road object in the direction of the lane or road_block. As road_segments do not have a direction (e.g. intersections), we return all possible next roads.

We can also visualize the next roads using the render_next_roads(x, y) function. We see that there are 3 adjacent roads to the intersection specified by (x, y).

Working with Lanes

For the prediction challenge we added connectivity information to the map expansion (v1.2) to efficiently query which lane is connected to which other lanes. Below we render the lane and lane_connector objects. The lanes and lane_connectors are defined by parametric curves. The resolution_meters parameter specifies the discretization resolution of the curve. If we set it to a high value (e.g. 100), the curves will appear as straight lines. We recommend setting this value to 1m or less.

To get the closest lane to a location, use the get_closest_lane method. To see the internal data representation of the lane, use the get_lane_record method. You can also explore the connectivity of the lanes, with the get_outgoing_lanes and get_incoming_lane methods.

To help manipulate the lanes, we've added an arcline_path_utils module. For example, something you might want to do is discretize a lane into a sequence of poses.

Given a query pose, you can also find the closest pose on a lane.

To find the entire length of the lane, you can use the length_of_lane function.

You can also compute the curvature of a lane at a given length along the lane.

Data Exploration

Let's render a particular patch on the map:

A lot of layers can be seen in this patch. Lets retrieve all map records that are in this patch.

Since there are a lot of records, we focus only on the layer road_segment:

We see that using the option intersect typically returns more records than within.

Looking at the above plot. Point (390, 1100) seems to be on a stop line. Lets verify that.

Indeed, we see a stop_line record.

To directly check for stop_line records, we run:

Let's look at the bounds/extremities of that record

Layers

Let us look more closely at the different map layers:

Our map database consists of multiple layers. Where each layer is made up of records. Each record will have a token identifier.

We see how our map layers are divided into two types of layers. One set of layer belong to the geometric_layers group, another set of layers belongs to the non_geometric_layers group.

  1. geometric_layers define geometric entities in the map:
    • Nodes (2d points) are the basis for all geometric layers.
    • Lines consist of two or more nodes. Formally, one Line record can consist of more than one line segment.
    • Polygons consist of three or more nodes. A polygon can have holes, thus distorting its formal definition. Holes are defined as a sequence of nodes that forms the perimeter of the polygonal hole.
  1. non_geometric_layers represent physical entities in the map. They can have more than one geometric representation (such as drivable_areas), but must be strictly of one type (e.g. road_segment, lane_divider).

1. Geometric layers

a. Node

The most primitive geometric record in our map database. This is the only layer that explicitly contains spatial coordinates.

b. Line

Defines a line sequence of one or more lines and therefore consists of two or more nodes.

c. Polygon

Defines a polygon which may contain holes.

Every polygon record comprises of a list of exterior nodes, and zero or more list(s) of nodes that constitute (zero or more) holes.

Let's look at one polygon record:

2. Non geometric layers

Every non-geometric layer is associated with at least one geometric object.

a. Drivable Area

Drivable area is defined as the area where the car can drive, without consideration for driving directions or legal restrictions. This is the only layer in which the record can be represented by more than one geometric entity. Note: On some machines this polygon renders incorrectly as a filled black rectangle.

b. Road Segment

A segment of road on a drivable area. It has an is_intersection flag which denotes whether a particular road segment is an intersection.

It may or may not have an association with a drivable area record from its drivable_area_token field.

As observed, for all non geometric objects except drivable_area, we provide a shortcut to its nodes.

Let's take a look at a road_segment record with is_intersection == True

If we render this road segment we can see that it is indeed an intersection:

c. Road Block

Road blocks are blocks of a road that have the same traffic direction. Multiple road blocks are grouped in a road segment.

Within a road block, the number of lanes is consistent.

Every road block has a from_edge_line_token and to_edge_line_token that denotes its traffic direction.

d. Lanes

Lanes are parts of the road where vehicles drive in a single direction.

Aside from the token and the geometric representation, a lane has several fields:

e. Pedestrian Crossing

Pedestrian crossings are regions where pedestrians can legally cross the road, typically highlighted by white markings. Each pedestrian crossing record has to be on a road segment. It has the road_segment_token field which denotes the road_segment record it is associated with.

f. Walkway

A walkway or sidewalk is the typically elevated area next to a road where pedestrians are protected from vehicles on the road.

g. Stop Line

The physical world's stop line, even though the name implies that it should possess a line geometric representation, in reality its physical representation is an area where the ego vehicle must stop.

It has several attributes:

h. Carpark Area

A car park or parking lot area.

It has several attributes:

i. Road Divider

A divider that separates one road block from another.

road_segment_token saves the association information to a road_segment.

j. Lane Divider

A lane divider comes between lanes that point in the same traffic direction.

The lane_divider_segments field consist of different nodes and their respective segment_types which denotes their physical appearance.

k. Traffic Light

A physical world's traffic light.

It has several attributes:

  1. traffic_light_type denotes whether the traffic light is oriented horizontally or vertically.
  2. from_road_block_tokens denotes from which road block the traffic light guides.
  3. items are the bulbs for that traffic light.
  4. pose denotes the pose of the traffic light.

Let's examine the items field

As mentioned, every entry in the items field is a traffic light bulb. It has the color information, the shape information, rel_pos which is the relative position, and the to_road_block_tokens that denotes to which road blocks the traffic light bulb is guiding.