Coupled simulator for research on driver-pedestrian interactions made in Unity.
Usage of the simualator
The simulator is open-source and free to use. It is aimed for, but not limited to, academic research. We welcome forking of this repository, pull requests, and any contributions in the spirit of open science and open-source code 😍😄 For enquiries about collaboration, you may contact email@example.com.
If you use coupled-sim for academic work please cite the following paper.
Bazilinskyy, P., Kooijman, L., Dodou, D., & De Winter, J. C. F. (2020). Coupled simulator for research on the interaction between pedestrians and (automated) vehicles. Under review.
Description of the simulator
The coupled simulator supports both day and night-time settings. Figure above shows a view of the night mode. Figure below shows the top view of the environment. It is a model of a city centre containing:
- Network of 2-lane roads
- 10 intersections with traffic lights that can be turned on and off before the experiment or programmatically in real-time
- 34 zebra crossings
- Static objects (buildings, parked cars, trees)
- Advertisements (programmable and can be used as visual distractions)
- small (similar to Smart Fortwo)
- medium (similar to Pontiac GTO)
- large (similar to Nissan Datsun)
Cars that are not controlled by the human participants can be instructed to follow a trajectory before the experiment or can be programmed to respond to other road users.
The coupled simulator supports a keyboard and a gaming steering wheel as input sources for the driver of the manual car, a keyboard for the passenger of the AV to control the external human-machine interface, and a motion suit for the pedestrian. At the moment, supported motion suit is Xsens Motion Suit.
The supported sources of output are a head-mounted display (HMD) and computer screen for the driver, a computer screen for the passenger, and a head-mounted display for the pedestrian. At the moment, supported HDM is Oculus Rift CV1.
Networking and data logging
The current number of human participants supported by the coupled simulator is three. However, this number can be expanded up to the number of agents supported by the network. Synchronisation in a local network is handled by a custom-made network manager designed to support the exchange of information between agents with low latency and real-time data logging at 50 Hz for variables from the Unity environment and up to 700Hz from the motion suit. The data that are logged include the three-dimensional position and rotation of the manual car and the AV, the use of blinkers by the driver of the manual car, and 150 position and angular variables from the motion suit. The data are stored in binary format, and the coupled simulator contains a function to convert the saved data into a CSV file. The host agent initiates a red bar that is displayed across all agents for 1 s to allow for visual synchronisation in case screen capture software is used.
The simualator was tested on Windows 10 and macOS Mojave. All functionality is supported by both platforms. However, support for input and output devices was tested only on Windows 10.
After checking out this project, launch Unity Hub to run the simulator with the correct version of Unity (currently 2018.4.6f1).
How to run
Select the project from the Unity Hub projects list. Wait until the project loads in. If it is not in the Unity Hub list (it is the first time you are running the project), it has to be added first - click Add and select a folder containing the project files. Once the project is loaded into the Unity editor press the Play button to run it.
Steps to run an experiment:
- Start host and wait for clients to join if needed. To start host press Start Host button. To start the client press Start Client button, enter the host IP address and press Connect.
- Once all clients have joined, on the host, select one of the experiments listed under Experiment:.
- On the host, assign roles to participants.
- On both host and clients, each participant has to select control mode.
- Start an experiment with the Start Game button.
The central point for configuring the simulator is Managers game object from the StartScene scene. It has two components:
- PlayerSystem: gathering references to player avatar prefabs,
- NetworkingManager: gathering references to experiment definitions and elements spawned during networked experiment runtime (currently only waypoint-tracking cars - AICar).
The experiment is defined solely with prefab containing the ExperimentDefinition component in the root object. To edit the experiment definition, double click the prefab in the Project window. To make newly created experiment selectable you have to add its prefab to Experiments list on NetworkingManager component.
Prefab will be opened in edit mode along with the currently defined Regular Prefab Editing Environment. When defining the experiment it is worth setting Regular Prefab Editing Environment variable to the Unity scene defined in the experiment (Edit -> Project Settings -> Editor -> Prefab Editing Environments -> Regular Environment).
ExperimentDefinition component defines the following fields:
- Name: the name of the experiment
- Scene: Unity scene name to be loaded as an experiment environment
- Roles: list defining roles that can be taken during an experiment by participants
- Points of Interest: static points that are logged in experiment logs to be used in the log processing and analysis
- Car Spawners: references to game objects spawning non-player controlled cars
Points of interest is a list of Transform references. CarSpawners list references game objects containing component inheriting from CarSpawnerBase. It defines, with overridden IEnumerator SpawnCoroutine() method, spawn sequence (see TestSyncedCarSpawner for reference implementation). Car prefabs spawned by the coroutine with AICar Spawn(AICar prefab, bool yielding) method must be one of the referenced prefabs in AICarSystem list on NetworkManager component.
Configuration of agents
Roles field is a list of ExperimentRoleDefinition struct’s defining experiment roles with the following data fields:
- Name: short name/description of the role
- SpawnPoint.Point: defines where player avatar will be spawned
- SpawnPoint.Type: a type of player avatar. It may be either Pedestrian, Driver, Passenger of an autonomous car.
Adding and removing agents
To add a new agent either increase the size of Roles array or duplicate existing role by right-clicking on the role name and selecting Duplicate from the context menu.
To remove the agent right click on the role name and select Delete from the context menu or decrease the list size removing end entries that don’t fit the resized list.
Configuration of starting location of agents
Add a new game object to the prefab and set its position and rotation. Drag the newly created object to the SpawnPoint.Point in role definition.
Configuration of the pedestrian agent
No additional configuration is needed for pedestrian type agents.
Configuration of the driver agent
For Driver role following field has to be defined:
- CarIdx - points to a car prefab on the AvatarPrefabDriver (field on PlayerSystem component) list that will be spawned for this role.
Configuration of the passenger agent
For Passenger type of agent following additional fields has to be defined:
- Car Idx - indicates car prefab that will be spawned for this role. Selected prefab is the one on the indicated index on PlayerSystem lists (for PassengerAvatarPrefabPassenger list)
- Three DriverHMI fields - define which HMI prefab to spawn on indicated spots
- AutonomusPath - references game object defining waypoints for the autonomous car via WaypointCirciut component
Configuration of non-playable characters
Paths for both non-playable pedestrians and vehicles are defined with WaypointCircuit component. To add waypoint press plus sign and drag waypoint Transform into the newly added field. To remove waypoint press a minus sign next to the waypoint. To reorder waypoint click up/down signs next to a waypoint. To change the position of a waypoint select waypoint transform and move it do the desired position.
Configuration of the movement of the non-playable vehicles
Additionally, for vehicles, SpeedSetting along with Collider component might be used to further configure tracked path.
Configuration of daylight conditions
DayNightControl component helps to define different experiment daylight conditions. It gathers lighting-related objects and allows defining two lightings presets - Day and Night, that can be quickly switched for a scene. This component is intended to be used at the experiment definition setup stage. When the development of the environment is complete, it is advised, to save the environment into two separate scenes (with different light setups) and bake lightmaps.
Configuration of traffic lights
Creating a traffic street lights system is best started with creating an instance of ExampleStreetLightCrossSection and adjusting it. Traffic light sequence is defined in StreetLightManager component as a list of StreetLightEvents. Events are processed sequentially. Each event is defined with the following fields:
- Name: descriptive name of an event
- Delta Time: relative time that has to pass since previous event to activate the event
- CarSections: cars traffic light group that the event applies to
- PedestrianSections: pedestrian traffic light group that the event applies to
- State: state to be set on the lights specified by sections, LOOP_BACK is a special state that restarts the whole sequence
Details on editing car prefabs
Speedometer component controls a speed indicator. To use digital display, set the Speedometer Text field in order to do that. To use analog display, set the following fields:
- Pivot - the pivot of an arrow
- PivotMinSpeedAngle - the inclination of an arrow for 0 speed
- PivotMaxSpeedAngle - the inclination of an arrow for max speed
- MaxSpeed - max speed displayed on the analog display
Troubleshooting MVN suit
Running the simulation
Connect the Asus Router with the USB Adapter to the PC and plug the Ethernet in a yellow port of the router. Let it boot up for some time while you prepare the software and the suit. Connect the Oculus Rift HDMI and USB 3.0 to the computer. Plug in the black MVN dongle in a USB 3.0 port of the computer Run the latest version of MVN Analyze. Fill out the anthropometric data and participant name by creating a new recording (1).
Check under Options -> Preferences -> Miscellaneous -> Network Streamer that the data streamer is set to Unity3D.
Note: if you find out later on that somehow the avatar looks buggy in Unity, play around with the down sampling skip factor in the Network Streamer overview to improve rendering.
Turn on the Body Pack with all the sensors connected. This will sound like one beep. Wait for 30 seconds and press the WPS button on the router. When the Body Pack of the suit beeps again, continuously press the button for 2 seconds until you hear two sequential beeps. Look if the light of the button becomes a strobe light. If this does not happen within a minute or 5, you’ve got a problem named Windows.
The problem named Windows
Delete the following software from the pc and re-install the latest version of MVN Analyze. This will re-install also the bonjour print services.
The post era of having problems with Windows
If you have an avatar in MVN Analyze and all the sensors are working, boot Unity for the simulation. See figure below: press play to launch the simulation, use the dropdown menu to select a participant and press the trial button to launch a trial.
Note. If you want to match the orientation of the Oculus to the orientation of the avatar’s head, make sure you have left-clicked the game screen in Unity and press the R-key on your keyboard. Pressing the R-key to match the visuals with the head orientation is an iterative process which requires feedback from the participants.
If you want to start a new trial, click play again at the top of the Unity screen to end the simulation. Also match the head orientations again with the R-key loop.
Troubleshooting Oculus Rift
Always run Oculus Home software when using Oculus Rift. Otherwise you will encounter black screens. Make sure your graphics driver and USB 3.0 connections are up to date. If Oculus gives a critical hardware error, disconnect Rift and set Oculus Home software to Beta (Public Test Channel, 1st figure below) and check if Oculus Home setting is set to allow Unknown apps (2nd figure below).
The agent PCs need to be connected via a local network. If you cannot reach the host machine, try to ping it.
Inbound rules: Set correct Unity Editor version to allow all connections.
Troubleshooting steering wheel
Check if supporting software is installed (e.g., Logitech gaming software G27 is used in our case). In Unity, you can check which button corresponds to your specific wheel. You can find this out by using the following asset in an empty project: https://assetstore.unity.com/packages/tools/input-management/controller-tester-43621
Then make sure you assign the correct inputs in Unity under Edit -> Project Settings -> Input (see figure below).
We have used the following free assets:
|Textures||Various||Creative Commons CC0|
|Oculus Integration||Oculus||Oculus SDK License|
|Simple modular street Kit||Jacek Jankowski||Unity asset|
|Realistic Tree 10||Rakshi Games||Unity asset|
|Small Town America - Streets||MultiFlagStudios||Unity asset|
|Cars Free - Muscle Car Pack||Super Icon LTD||Depricated, Unity asset|
|Mini Cargo Truck||Marcobian Games||Unity asset|
|Street Bench||Rakshi Games||Unity asset|
|waste bin||Lowpoly_Master||Unity asset|
|Smart fortwo||Filippo Citati||MIT|