Simulation - A XR Music Mixer Journey
Role : Solo Project, Game Design, Program, Environment/UX Design, Asset Modeling
Tech Detail : Unity, C#, Blender
Timeframe : 3 Months, Janurary - April 2023
Tech Detail : Unity, C#, Blender
Timeframe : 3 Months, Janurary - April 2023
Overview
Dive into an odyssey of - spaceship DJ!
Yes, that’s a real thing. It’s where your controls of the spaceship also changes a mysterious music in the background.
Dive into an odyssey of - spaceship DJ!
Yes, that’s a real thing. It’s where your controls of the spaceship also changes a mysterious music in the background.
Task Overview
- Designed and implemented VR spaceship driving controls.
- Created and tuned UI and VFX to fit within the overall aesthetic and made sure players are comfortable within the VR environment.
- Implemented non-realtime audio spectrum analyzer using multi-threading solution.
- Asset modeling + texture creation and management for reusability.
System + Interaction Design
Abstracting A DJ-table
The core gameplay of DriverJ is based on the
To maximize the experience of DJ-ing while keeping the controls at an intuitive level, the DJ process is extracted into several key actions:
Play next song
Change both track speed separartely
Echo control
Duck volume (muffle audio) control
Volume control
Change both track speed separartely
Echo control
Duck volume (muffle audio) control
Volume control
Warp jump
Thrust/Up Thrust speed
Ship roll degree
Ship pitch degree
Ship Yaw degree
Thrust/Up Thrust speed
Ship roll degree
Ship pitch degree
Ship Yaw degree
Warp Jump
Echo/Roll cntrol
Second Track Playback speed/Thrust
Separate playback speed control reflected in sound spectrum display
TECHNICAL DETAIL
Spectrum design
A key element of any DJ table is the spectrum display of the tracks (usually two), such that the DJ can easily tone the tracks and perform tricks like sync, unsync, blend, etc.
The spectrum displays shows the intensity and speed of a given sound, when it reaches the middle bar it will be played. Thus musics with same spectrum distribution are said to have the same beats.
A key element of any DJ table is the spectrum display of the tracks (usually two), such that the DJ can easily tone the tracks and perform tricks like sync, unsync, blend, etc.
The spectrum displays shows the intensity and speed of a given sound, when it reaches the middle bar it will be played. Thus musics with same spectrum distribution are said to have the same beats.
challenges
To make sure the spectrum system mimics the real DJ table’s behavior, the spectrums must be analyzed before it is played, so player can see the coming beats at the next few moments and adjust accordingly.
Since the spectrum pre-analyze feature was not available in Unity’s sound processor (Unity only performs real-time analysis), it must be implemented. But because the songs are preloaded automatically when the previous song finished, and Unity does not starts the player on the main thread, it requiredmulti-threading solution to make sure the spectrum is pre-analyzed.
To make sure the spectrum system mimics the real DJ table’s behavior, the spectrums must be analyzed before it is played, so player can see the coming beats at the next few moments and adjust accordingly.
Since the spectrum pre-analyze feature was not available in Unity’s sound processor (Unity only performs real-time analysis), it must be implemented. But because the songs are preloaded automatically when the previous song finished, and Unity does not starts the player on the main thread, it required
The spectrum of the next song is pre-loaded into it’s slot, waiting for player activation