A Collaborative Mixed Reality Visualization Framework for Immersive Analytics
- Recommended Unity version: 2018.3.14f1
- Microsoft Mixed Reality Toolkit v2.0.0 RC2.1 (already included in the project)
- Photon PUN 2 (download through Unity’s asset store window in the editor)
- Visual Studio (tested with VS 2017)
- For HoloLens Development:
- Install Unity and Visual Studio
- Clone the project
- Start Unity, select open and choose the folder Frontend/VIAProMa in the project files
- Once the project has been loaded, go to Window > Asset Store (or press Ctrl + 9). Search and download “PUN 2 - FREE” by Exit Games. To do so, the asset has to be aquired first, then downloaded and then imported.
- Once PUN has been downloaded, an import window will appear. Select everything and click on import.
- After the successful import of the assets, a window should appear which asks for an app ID.
- Create a free Photon Engine accout under https://www.photonengine.com/en/pun.
- Register a Photon PUN application in order to generate an app ID.
- Enter the app ID in the dialog window in Unity. If the window has not appeared or has already been closed, go to Window > Photon Unity Networking > Highlight Server Settings and enter the app ID in the inspector under “Settings/ App Id Realtime”.
- Again, go to the Asset Store and download “Photon Voice 2” by Exit Games.
- In the import window, select everything that is selectable and click on import. Some parts of the import are not selectable. This is fine since they have already been imported with the Photon PUN 2 package.
- In the the dashboard of the Photon Engine account, create a register a new Photon Voice application and copy its app ID.
- After the import, go to Window > Photon Unity Networking > Highlight Server Settings and enter the voice app ID under “Settings/ App Id Voice”.
- Open the scene “Main Scene”. A prompt will appear which will ask to import TMP Essentials. Click on the upper button.
- (Optional) It makes sense to scale the editor icons down. This can be done under “Gizmos” in the top right of the 3D view. Pull the top most slider next to “3D Icons” down until the icons in the scene have the right size
- Run the application once in the editor in order to initialize the collection which keeps track of networked objects.
For Android development, additional steps can be found in ARCore’s quickstart guide.
In addition to the working logic, each of the project’s features has been isolated into its own scene which contains a minimal working example for the feature. This way, the functionality can be tested and new developers can experiment with these features in order to learn how they work. The scenes are situated in the folder “Tests”.
The backend folder contains a las2peer project which realizes a RESTful service.
To install the backend the following technologies are required.
- Java 8 (on Windows: make sure that the Java installation is placed in your path variables); if you type java –version in the command line, the output should similar to this:
java version "1.8.0_231" Java(TM) SE Runtime Environment (build 1.8.0_231-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode)
- Apache Ant (on Windows: make sure that the Ant installation is placed in your path variables; if you type ant –version in the command line, you should get an output similar to this:
Apache Ant(TM) version 1.10.1 compiled on February 2 2017
To develop the backend, you should also install an IDE, e.g. IntelliJ.
Building the Backend
You first have to fetch the dependencies by running
ant get_deps in the backend folder where the build.xml is stored.
After that, run
ant all in the backend folder.
Running the Backend
After the build, execute the corresponding “start_network” script in the “bin” folder of the backend.
It contains two scripts “start_network.bat” and “start_network.sh”. On Windows, execute the “start_network.bat” file. On Linux or Mac, first go back to the backend folder and execute
./bin/start_network.sh from there.
Problem: When building the application, errors are shown that the namespace name ‘HandJointKind’, ‘HandMeshObserver’ and ‘JointPose’ are not found.
Solution: Download the Windows 10 SDK 10.0.18362.0. After that, open the Build Settings Window and change the Target SDK Version and Minimum Platform Version to 10.0.18362.0.
Problem: The shared room which was created in one app instance does not appear on the other app instance.
Solution: Make sure that both app instances use the same gameVersion which is specified in the launcher script. (Assets/Scripts/Multiplayer/Launcher.cs) Also ensure that both app instances use the same Photon PUN version. The PUN version can be seen in the Photon server settings. Additionally, both instances must have the same Photon app-IDs in the server settings.