Core

class vista.core.World.World(trace_paths: List[str], trace_config: Optional[Dict] = {})[source]

This class specifies the World where all entities (i.e., agents and sensors) in VISTA lives in. The is built upon one or multiple traces pre-collected from the real world. Users can spawn agents (Car) in the World, where all agents will be visible while rendering sensory measurement. At reset, the World randomly sample an pointer in the dataset (conceptually retrieving a frame within a trace) as the initial condition for the simulation.

Parameters

  • trace_paths (List(str)) – A list of paths to traces

  • trace_config (Dict) – Configuration of traces

Example usage:

>>> world = vista.World(trace_paths, trace_config)
>>> agent = world.spawn_agent(agent_config)
>>> world.reset()
spawn_agent(config: Dict) → vista.entities.agents.Car.Car[source]

Spawn an agent in this world.

Parameters

config (Dict) – Configuration of the agent.

Returns

The agent being spawned.

Return type

Car

reset(initial_dynamics_fn: Optional[Dict[str, Callable]] = {}) → None[source]

Reset the world. This includes (1) sample a new anchor point from the real-world dataset to be simulated from and (2) reset states for all agents.

Parameters

initial_dynamics_fn (Dict[str, Callable]) – A dict mapping agent names to a function that initialize agents poses.

sample_new_location() → Tuple[int, int, int][source]

Sample a pointer to the dataset for simulation.

Returns

Return a tuple (int_a, int_b, int_c), where int_a is trace index, int_b is segment index, and int_c is frame index.

sample_new_trace_index() → int[source]

Sample a new trace index based on number of frames in a trace.

Returns

An index to specify which trace to be simulated from.

Return type

int

set_seed(seed) → None[source]

Set random seed.

Parameters

seed (int) – Random seed.

property seed

Random seed for sampling pointer to the dataset during reset().

property traces

All trace attached to the World.

property agents

All agents that live in the World.

class vista.core.Trace.Trace(trace_path: str, trace_config: Optional[Dict] = {})[source]

This is a helper class for handling a trace in the dataset. It allows straightforward access to any pointer in the trace. A trace is separated into several segments based on timestamps (split to segments if the difference of two consecutive timestamps is larger than a value). Each segment contains frames that can be accessed through specifying frame index. Note that frame index used in VISTA is not necessarily the same as frame number in the dataset. Overall, we use segment_index, frame_index, frame_number, timestamp in this class and there are several functions for conversion among them. The Trace also performs filtering including,

  • Filter frames based on annotation label (video_label.csv) if there is any.

  • Filter by end-of-trace and time difference across consecutive frames.

This class also handles sampling of pointer to the trace, i.e., sampling segment and frame index.

Parameters

  • trace_path (str) – Path to a trace.

  • trace_config (dict) – Configuration of the trace object.

find_segment_reset() → int[source]

Sample a segment index based on number of frames in each segment. Segments with more frames will be sampled with a higher probability.

Returns

Index of a segment.

Return type

int

find_frame_reset(segment_index: int) → int[source]

Sample a frame index in a segment.

Parameters

segment_index (int) – Index of the segment to be sampled from.

Returns

Frame index.

Return type

int

Raises

NotImplementedError – Unrecognized trace reset mode.

get_master_timestamp(segment_index: int, frame_index: int, check_end: Optional[bool] = False) → Union[float, Tuple[bool, float]][source]

Get timestamp from the master sensor.

Parameters

  • segment_index (int) – Segment index.

  • frame_index (int) – Frame index.

  • check_end (bool) – Whether to check if the given segment and frame index exceed the end of the trace; default to False.

Returns

Return a tuple (bool_a, float_a), where bool_a is whether the given segment and frame index exceed the end of the trace and float_a is timestamp; otherwise timestamp (float).

get_master_frame_number(segment_index: int, frame_index: int, check_end: Optional[bool] = False) → Union[int, Tuple[bool, int]][source]

Get frame number from the master sensor.

Parameters

  • segment_index (int) – Segment index.

  • frame_index (int) – Frame index.

  • check_end (bool) – Whether to check if the given segment and frame index exceed the end of the trace; default to False.

Returns

Return a tuple (bool_a, int_a), where bool_a is whether the given segment and frame index exceed the end of the trace and int_a is frame number; otherwise frame number (int).

set_seed(seed) → None[source]

Set random seed.

Parameters

seed (int) – Random seed.

property seed

Random seed for sampling pointer in find_segment_reset() and find_frame_reset().

property trace_path

Path to the directory that contains all data of this trace.

property multi_sensor

MultiSensor object. Please check the definition of this class.

property good_frames

A dictionary of good frame numbers, where keys are sensor names and values are lists of frame numbers (int).

property good_timestamps

A dictionary of good timestamps, where keys are sensor names and values are a list of timestamps (float).

property num_of_frames

Number of good frames.

property f_curvature

A 1D interpolation function for curvature of the ego-car.

property f_speed

A 1D interpolation function for speed of the ego-car.

property reset_mode

Trace reset mode.

property road_width

Road width.

property param_file

Path to the sensor parameters file for this trace.

class vista.core.Display.Display(world: vista.core.World.World, fps: Optional[float] = 30, display_config: Optional[Dict] = {})[source]

This is a visualizer of VISTA simulator. It renders an image that contains visualization of all sensors from all agents and a top-down view that depicts road and all cars in the scene within a predefined range based on the state of the simulator (World).

Parameters

  • world (vista.core.World) – World to be visualized.

  • fps (int) – Frame per second.

  • display_config (Dict) – Configuration for the display (visualization).

Raises

AssertionError – Grid spec is inconsistent with maximal number of sensors across agents.

Example usage:

>>> display_config = {
    'road_buffer_size': 200,
    'birdseye_map_size': (30, 20), # size of bev map in vertical and horizontal directions
    'gs_bev_w': 2, # grid spec width for the birdseye view block
    'gs_agent_w': 4, # grid spec width for an agent's block
    'gs_h': 6, # grid spec height
    'gui_scale': 1.0, # a global scale that determines the size of the figure
    'vis_full_frame': False, # if Display should not crop/resize camera for visualization purposes
}
>>> display = Display(world, )
reset() → None[source]

Reset the visualizer. This should be called every time after World reset. It basically reset the cache of road data used in the top-down view visualization.

render()[source]

Render an image that visualizes the simulator. This includes visualization of all sensors of every agent and a top-down view that depicts the road and all cars in the scene within a certain range. Note that it render visualization based on the current status of the world and should be called every time when there is any update to the simulator.

Returns

An image of visualization for the simulator.

Return type

np.ndarray

property ref_agent

Agent as a reference to compute poses of objects (e.g., cars, road) in visualization.

class vista.core.core_utils.MultiSensor(trace_dir: str, master_sensor: Optional[str] = 'camera_front')[source]

This class handles synchronization across multiple sensors in a trace. It basically reads timestamps files associated with every sensors existing in the trace and construct several helper functions for conversion between an unified timestamp used in the simulator and timestamp (or frame number, etc) of different sensors.

Parameters

  • trace_dir (str) – Directory to a trace.

  • master_sensor (str) – Name of the master sensor.

Raises

  • AssertionError –

    1. The name of the master sensor not included in a predefined

  • set of topic names specified in (2) No timestamp data for the master sensor. –

get_time_from_frame_num(sensor: str, frame_num: int) → float[source]

Compute the timestamp associated to a frame in a video return None if we dont have information about that frame.

Parameters

  • sensor (str) – Sensor name.

  • frame_num (int) – Frame number.

Returns

Timestamp associated with the given sensor and frame number.

Return type

float

get_frames_from_times(timestamps: List[float], fetch_smaller: Optional[bool] = False) → Dict[str, List[int]][source]

Takes in a list of timestamps and returns corresponding frame numbers for each sensor. Note that since sensors are not necessarily sync’ed, the returned frame numbers are the one with the closest (smaller) timestamps.

Parameters

  • timestamps (list) – A list of timestamps.

  • fetch_smaller (bool) – Whether to fetch the closes and smaller timestamps.

Returns

Corresponding frame numbers for all sensors.

Return type

dict

get_master_timestamps() → List[float][source]

Get all timestamps of the main sensor.

Returns

A list of timestamp.

Return type

list

set_main_sensor(sensor_type: str, sensor_name: str) → None[source]

Set main sensor based on sensor’s type and name.

Parameters

  • sensor_type (str) – Type of the sensor to be set (camera, lidar, or event camera).

  • sensor_name (str) – Name of the sensor to be set.

property sensor_names

A list of all sensors’ names.

property camera_names

A list of RGB cameras’ names.

property main_camera

The main RGB camera object.

property lidar_names

A list of LiDARs’ names.

property main_lidar

The main LiDAR object

property main_event_camera

The main event camera object.

property master_sensor

The name of the master sensor.

class vista.core.core_utils.LabelSearch(time_of_day: str, weather: str, road_type: str, maneuver: str, direction: str, tag: str)[source]

This class handles annotations of the collected traces and filter out frames that are not good based on several fields including time of day, weather, road type, maneuver, direction, tag. Please check data annotation process for more details on each field.

Parameters

  • time_of_day (str) – Time of day to be considered as good frames.

  • weather (str) – Weather condition to be considered as good frames.

  • road_type (str) – Road type to be considered as good frames.

  • maneuver (str) – Human maneuver to be considered as good frames.

  • direction (str) – Direction to be considered as good frames.

  • tag (str) – Other tags to be considered as good frames.

find_good_labeled_frames(trace_path: str) → Tuple[numpy.ndarray, numpy.ndarray][source]
Find good frames based on video labels. Assume video labels have

consistent timestampswith the specified master sensor/topic.

Parameters

trace_path (str) – path to a trace

Returns

Return a tuple (arr_1, arr_2), where arr_1 is a boolean mask of good frames, and arr_2 is timestamps of all good frames.

class vista.core.core_utils.TopicNames[source]

This class specifies (ROS) topic names of each sensors.