Published by Addison-Wesley Professional (July 14, 2021) © 2019
Sam KeeneThe Complete, Up-to-Date Guide to Building AR and VR Games
Keene also provides an extensive library of downloadable, up-to-the-minute ARCore and Daydream code to jumpstart your project. In addition, he takes you through crucial UX design principles and best practices learned from building large scale VR and AR apps at Google.
Google Daydream VR Cookbook shows you how to:
Whether you are a software developer, UX professional, visual designer, beginner, or you come from a different design field, this book is a great practical introduction to VR and AR.
Google’s new ARCore and Daydream VR platforms enable you to deliver advanced augmented and virtual reality games and apps on a wide spectrum of modern Android devices. Now for the first time, there’s a comprehensive deep dive into both ARCore and Daydream for every Android developer and designer. Multi-award-winning AR/VR developer Sam Keene takes a hands-on approach, leading you through all aspects of the ARCore and Daydream frameworks and SDKs, with step-by-step tutorials and advice for building pro-quality AR/VR games and apps.
Keene presents his material as a cookbook of recipes to get you up and running with VR/AR development as fast and as painlessly as possible. The recipes in most chapters start by assembling the essential building blocks, which are pieced together to create something larger. You are then free to take these building blocks and turn them into your own creation.
Google Daydream VR Cookbook shows you how to:
- Install and explore the Google Daydream development tools
- Master basic and advanced Daydream Controller techniques
- Implement intuitive VR user interfaces
- Integrate audio, video, and realistic physics into your VR games
- Install and explore the ARCore SDK and development tools
- Learn how to build AR apps that solve real user needs
- Master AR game development using ARCore
- Optimize VR and AR game performance
Whether you are a software developer, UX professional, visual designer, beginner, or you come from a different design field, this book is a great practical introduction to VR and AR.
Preface xiii
Chapter 1 VR and AR 1
Immersive Computing 2
VR and AR 2
Media Archaeology 3
Stereoptics 4
Pre-Cinema Animation 6
Computing 9
Twentieth Century VR 10
Google VR and AR 11
Daydream VR 12
ARCore 14
Summary 15
Chapter 2 Daydream and Unity 17
Getting to Know Unity 18
What Is Unity? 18
Why Unity? 19
Exploring the Unity Editor 19
Exploring Unity Concepts 23
GameObjects 23
Components 24
Prefabs 24
Scripts 24
MonoBehaviour 25
Accessing Variables in the Inspector 26
Daydream Development 26
Daydream Development Environment 27
Daydream Hardware 27
Google VR SDK 28
Setting Up Your Daydream Developer Environment 29
Installing the Android SDK and Java JDK in Unity 30
Installing the Google VR SDK for Unity 31
Troubleshooting Setup and Installation Issues 32
Daydream Elements 33
Recipe 2.1: Installing Elements 34
Recipe 2.2: Setting Up Daydream Instant Preview 35
Recipe 2.3: Setting Up a Daydream Controller Emulator 36
Recipe 2.4: VR Developer Options 37
Core Daydream Components 38
GvrEditorEmulator (Prefab) 38
GvrControllerMain (Prefab) 39
GvrControllerPointer (Prefab) 39
GvrEventSystem (Prefab) 39
GvrPointerPhysicsRaycaster (Script) 39
GvrPointerGraphicsRaycaster (Script) 40
Setting Up a Player in VR 40
Recipe 2.5: Building a VR Player in the First Person 41
Recipe 2.6: Traveling via Teleportation 43
Recipe 2.7: Customizing the Teleportation System 45
Recipe 2.8: Setting Up the Chase Camera 45
Recipe 2.9: Using a Tunneling Effect to Combat Motion Sickness 48
Summary 52
Chapter 3 Getting to Know the Daydream Controller 53
Getting to Know the Daydream Controller 54
Why the Controller? 54
How the Controller Works 54
Controller Support 58
Controlling the Controller 60
Handling the Controller API 61
Basic Interaction 66
Visualizing the Controller 78
Summary 89
Chapter 4 Building UI in VR 91
Designing User Interfaces in VR 92
User Experience Design in VR 93
Ergonomics 94
Daydream’s Design System: dmm 95
UI, Unity, and the Google VR SDK 97
Canvas 98
Rect Transform 99
Rect Tool 99
GvrPointerGraphicsRaycaster 100
Building the UI 100
Recipe 4.1: Adding a Canvas in dmms 101
Recipe 4.2: Adding an Image 103
Recipe 4.3: Adding Text 105
Recipe 4.4: Adding a Button 107
Recipe 4.5: Implementing a Game Menu 110
UI Layouts in VR 117
Recipe 4.6: Creating an Animated Cell 117
Recipe 4.7: Cell Tilt and Hover Effect 120
Recipe 4.8: Cell Masking Animation Effect 125
Recipe 4.9: Horizontal Layout 129
Recipe 4.10: Nested Layouts 131
Recipe 4.11: Grid Layout 133
Recipe 4.12: Multipanel Scrolling Layout 135
Summary 136
Chapter 5 Video and Audio 137
Video in Daydream 138
Video in VR 138
Streaming Video 141
Daydream and Video 141
Audio in Daydream 161
Summary 172
Chapter 6 Physics Games 173
Physics, Games, Daydream, and Unity 174
Unity Physics 174
Rigidbody Component 174
Colliders 176
Handling Collision Events 176
Fixed Update 177
Profiling 177
Building the Games 178
Recipe 6.1: Intro to Physics—Flipping Pancakes 179
Recipe 6.2: Building a Pop Gun Mechanic 186
Recipe 6.3: Building a Grenade Mechanic 188
Recipe 6.4: Weapon Selector 192
Building an Alien Shooter Game 193
Recipe 6.5: Building the Core Mechanic 193
Turning It into a Game 200
Recipe 6.6: Building the UI and Finishing the Game 210
Where to Go from Here 217
Summary 217
Chapter 7 ARCore 219
Augmented Reality 220
AR and UX 220
Lessons, Principles, and Practice 221
Key Technologies in ARCore 225
Motion Tracking 226
Environmental Understanding 226
Light Estimation 227
ARCore and Unity 227
ARCore Classes 228
ARCore Unity Components 230
ARCore Recipes 231
Recipe 7.1 Installing and Running ARCore 232
Recipe 7.2 Building an ARCore Template Scene 234
Recipe 7.3 Selecting and Dragging Objects along a Plane 240
Recipe 7.4 Dragging Objects with Physics Weight 243
Recipe 7.5 Selecting an Object and Rotating It 251
Recipe 7.6: Creating a Target Below the Landing Position of the Dragged Object 253
Recipe 7.7 Retrieving All GameObjects Attached to an Anchor 257
Recipe 7.8 Retrieving All GameObjects Attached to a Trackable 258
Recipe 7.9 Hiding the Tracked Surface Visualization 258
Recipe 7.10 Getting the Y Position of the Lowest Plane 259
Recipe 7.11 Attaching Objects to Non-Horizontal Surfaces 260
Recipe 7.12: Customizing the Surface Visualization 260
Recipe 7.13 Pinching to Scale a GameObject 261
Recipe 7.14: Handling AR Tracking State 262
Summary 264
Chapter 8 AR Apps 265
AR Applications 266
Building User-Centered AR Apps 266
Recipe 8.1: Creating an AR Drawing App 266
ARCore and the Poly Toolkit 271
Poly Toolkit and Poly API 271
Recipe 8.2: Integrating the Poly Toolkit with ARCore and Unity 272
Recipe 8.3: Using Poly Toolkit Runtime 273
Recipe 8.4: Creating an AR Shopping Catalogue App 280
Summary 292
Chapter 9 AR Games 293
Types of AR Games 294
AR Game Lifecycle 295
Recipe 9.1 AR Tabletop Game 296
Building the Loader Scene 297
Building the Level Scene 305
Summary 325
Chapter 10 Optimization and Performance 327
Optimization Strategy 328
Isolating Bottlenecks 328
Fixing Bottlenecks 329
Profile Early, Profile Often 329
Tools for Analyzing and Optimizing 329
Analyzing 329
Optimizing: Daydream Renderer 331
Optimizing Frame Rate 333
Draw Calls 333
Texture Memory 334
Polygon Count: Geometry 335
Lighting 336
Culling 336
Other Tricks and Tips 337
Summary 338
Index 339
Chapter 1 VR and AR 1
Immersive Computing 2
VR and AR 2
Media Archaeology 3
Stereoptics 4
Pre-Cinema Animation 6
Computing 9
Twentieth Century VR 10
Google VR and AR 11
Daydream VR 12
ARCore 14
Summary 15
Chapter 2 Daydream and Unity 17
Getting to Know Unity 18
What Is Unity? 18
Why Unity? 19
Exploring the Unity Editor 19
Exploring Unity Concepts 23
GameObjects 23
Components 24
Prefabs 24
Scripts 24
MonoBehaviour 25
Accessing Variables in the Inspector 26
Daydream Development 26
Daydream Development Environment 27
Daydream Hardware 27
Google VR SDK 28
Setting Up Your Daydream Developer Environment 29
Installing the Android SDK and Java JDK in Unity 30
Installing the Google VR SDK for Unity 31
Troubleshooting Setup and Installation Issues 32
Daydream Elements 33
Recipe 2.1: Installing Elements 34
Recipe 2.2: Setting Up Daydream Instant Preview 35
Recipe 2.3: Setting Up a Daydream Controller Emulator 36
Recipe 2.4: VR Developer Options 37
Core Daydream Components 38
GvrEditorEmulator (Prefab) 38
GvrControllerMain (Prefab) 39
GvrControllerPointer (Prefab) 39
GvrEventSystem (Prefab) 39
GvrPointerPhysicsRaycaster (Script) 39
GvrPointerGraphicsRaycaster (Script) 40
Setting Up a Player in VR 40
Recipe 2.5: Building a VR Player in the First Person 41
Recipe 2.6: Traveling via Teleportation 43
Recipe 2.7: Customizing the Teleportation System 45
Recipe 2.8: Setting Up the Chase Camera 45
Recipe 2.9: Using a Tunneling Effect to Combat Motion Sickness 48
Summary 52
Chapter 3 Getting to Know the Daydream Controller 53
Getting to Know the Daydream Controller 54
Why the Controller? 54
How the Controller Works 54
Controller Support 58
Controlling the Controller 60
Handling the Controller API 61
Basic Interaction 66
Visualizing the Controller 78
Summary 89
Chapter 4 Building UI in VR 91
Designing User Interfaces in VR 92
User Experience Design in VR 93
Ergonomics 94
Daydream’s Design System: dmm 95
UI, Unity, and the Google VR SDK 97
Canvas 98
Rect Transform 99
Rect Tool 99
GvrPointerGraphicsRaycaster 100
Building the UI 100
Recipe 4.1: Adding a Canvas in dmms 101
Recipe 4.2: Adding an Image 103
Recipe 4.3: Adding Text 105
Recipe 4.4: Adding a Button 107
Recipe 4.5: Implementing a Game Menu 110
UI Layouts in VR 117
Recipe 4.6: Creating an Animated Cell 117
Recipe 4.7: Cell Tilt and Hover Effect 120
Recipe 4.8: Cell Masking Animation Effect 125
Recipe 4.9: Horizontal Layout 129
Recipe 4.10: Nested Layouts 131
Recipe 4.11: Grid Layout 133
Recipe 4.12: Multipanel Scrolling Layout 135
Summary 136
Chapter 5 Video and Audio 137
Video in Daydream 138
Video in VR 138
Streaming Video 141
Daydream and Video 141
Audio in Daydream 161
Summary 172
Chapter 6 Physics Games 173
Physics, Games, Daydream, and Unity 174
Unity Physics 174
Rigidbody Component 174
Colliders 176
Handling Collision Events 176
Fixed Update 177
Profiling 177
Building the Games 178
Recipe 6.1: Intro to Physics—Flipping Pancakes 179
Recipe 6.2: Building a Pop Gun Mechanic 186
Recipe 6.3: Building a Grenade Mechanic 188
Recipe 6.4: Weapon Selector 192
Building an Alien Shooter Game 193
Recipe 6.5: Building the Core Mechanic 193
Turning It into a Game 200
Recipe 6.6: Building the UI and Finishing the Game 210
Where to Go from Here 217
Summary 217
Chapter 7 ARCore 219
Augmented Reality 220
AR and UX 220
Lessons, Principles, and Practice 221
Key Technologies in ARCore 225
Motion Tracking 226
Environmental Understanding 226
Light Estimation 227
ARCore and Unity 227
ARCore Classes 228
ARCore Unity Components 230
ARCore Recipes 231
Recipe 7.1 Installing and Running ARCore 232
Recipe 7.2 Building an ARCore Template Scene 234
Recipe 7.3 Selecting and Dragging Objects along a Plane 240
Recipe 7.4 Dragging Objects with Physics Weight 243
Recipe 7.5 Selecting an Object and Rotating It 251
Recipe 7.6: Creating a Target Below the Landing Position of the Dragged Object 253
Recipe 7.7 Retrieving All GameObjects Attached to an Anchor 257
Recipe 7.8 Retrieving All GameObjects Attached to a Trackable 258
Recipe 7.9 Hiding the Tracked Surface Visualization 258
Recipe 7.10 Getting the Y Position of the Lowest Plane 259
Recipe 7.11 Attaching Objects to Non-Horizontal Surfaces 260
Recipe 7.12: Customizing the Surface Visualization 260
Recipe 7.13 Pinching to Scale a GameObject 261
Recipe 7.14: Handling AR Tracking State 262
Summary 264
Chapter 8 AR Apps 265
AR Applications 266
Building User-Centered AR Apps 266
Recipe 8.1: Creating an AR Drawing App 266
ARCore and the Poly Toolkit 271
Poly Toolkit and Poly API 271
Recipe 8.2: Integrating the Poly Toolkit with ARCore and Unity 272
Recipe 8.3: Using Poly Toolkit Runtime 273
Recipe 8.4: Creating an AR Shopping Catalogue App 280
Summary 292
Chapter 9 AR Games 293
Types of AR Games 294
AR Game Lifecycle 295
Recipe 9.1 AR Tabletop Game 296
Building the Loader Scene 297
Building the Level Scene 305
Summary 325
Chapter 10 Optimization and Performance 327
Optimization Strategy 328
Isolating Bottlenecks 328
Fixing Bottlenecks 329
Profile Early, Profile Often 329
Tools for Analyzing and Optimizing 329
Analyzing 329
Optimizing: Daydream Renderer 331
Optimizing Frame Rate 333
Draw Calls 333
Texture Memory 334
Polygon Count: Geometry 335
Lighting 336
Culling 336
Other Tricks and Tips 337
Summary 338
Index 339