Ar Uco Detection Holo Lens Unity by doughtmw - 4

VR & ARAnalytics

ArUco marker tracking on the HoloLens, implemented in Unity.

Unknown VersionUnknown LicenseUpdated 5 days agoCreated on November 20th, 2019
Go to source

ArUcoDetectionHoloLens-Unity

ArUco marker tracking made available for use in Unity through IL2CPP Windows Runtime support.

Incorporates:

aruco tracking

Requirements

ArUco Detection Sample

  1. Open HoloLensForCV sample in VS2017 and install included OpenCV.UWP.411 NuGet package to HoloLensForCV project
  2. Build the HoloLensForCV project (x86, Debug or Release)
  3. Copy all output files from HoloLensForCV output path (dlls and HoloLensForCV.winmd) to the Assets->Plugins->x86 folder of the ArUcoDetectionHoloLensUnity project
  4. Open ArUcoDetectionHoloLensUnity Unity project and build using IL2CPP, ensure that unsafe code is allowed under Unity Player Settings->Other Settings
  5. Open VS solution, build then deploy to device
  6. When looking at an aruco marker from the correct ArUco dictionary: this sample uses the 6x6 (50, 100, 250, 500) dictionary by default, a virtual cube, which is scaled to the size of the ArUco marker, should appear on top of the marker

aruco tracking view through pv camera

Figure is taken through the Microsoft Mixed Reality Capture interface, leading to a discrepancy in the position of the virtual cube. The first figure was recorded directly through the HoloLens optical display and is a better representation of how tracking looks in the point-of-view of a user.

Using other research mode sensor streams

To enable the use of HoloLens research mode streams in Unity, additional permissions are required for the project. Navigate to Unity project build folder and modify the Package.appxmanifest file to include:

  • Restricted capabilities package:
<Package 
  xmlns:mp="http://schemas.microsoft.com/appx/2014/phone/manifest" 
  xmlns:uap="http://schemas.microsoft.com/appx/manifest/uap/windows10" 
  xmlns:uap2="http://schemas.microsoft.com/appx/manifest/uap/windows10/2" 
  xmlns:uap3="http://schemas.microsoft.com/appx/manifest/uap/windows10/3" 
  xmlns:uap4="http://schemas.microsoft.com/appx/manifest/uap/windows10/4" 
  xmlns:iot="http://schemas.microsoft.com/appx/manifest/iot/windows10" 
  xmlns:mobile="http://schemas.microsoft.com/appx/manifest/mobile/windows10" 
  xmlns:rescap="http://schemas.microsoft.com/appx/manifest/foundation/windows10/restrictedcapabilities" 
  IgnorableNamespaces="uap uap2 uap3 uap4 mp mobile iot rescap" 
  xmlns="http://schemas.microsoft.com/appx/manifest/foundation/windows10"> 
  • Modified capabilities with with new package:
  <Capabilities>
    <rescap:Capability Name="perceptionSensorsExperimental" />
    <Capability Name="internetClient" />
    <Capability Name="internetClientServer" />
    <Capability Name="privateNetworkClientServer" />
    <uap2:Capability Name="spatialPerception" />
    <DeviceCapability Name="webcam" />
  </Capabilities>

Now, modifying the selected sensor for streaming in Unity (for depth and infrared streams, more processing of input frames will likely be required to visualize) different streams can be selected.

research mode sensor enum

Show all projects by doughtmw