Published by Addison-Wesley Professional (December 5, 2017) © 2018
Janie ClaytonMaster Metal: The Next-Generation Graphics and GPU Programming Platform for Apple Developers
Metal enables Apple developers to maximize performance in demanding tasks like 3D graphics, games, scientific programming, visualization, and GPU-accelerated machine learning. Metal® Programming Guide is the authoritative, practical guide to Metal for all iOS programmers who are interested in graphics programming but don’t know where to start.
Pioneering Apple developer Janie Clayton covers everything from basic draw calls to advanced parallel computing, combining easy-to-understand conceptual explanations with well-tested Swift 4/Xcode 9 sample code (available for download at GitHub).
Clayton introduces the essential Metal, graphics, and math concepts every graphics programmer needs to know. She also discusses key graphics-specific libraries, concepts, and Metal Classes, presenting techniques and examples you’ll find valuable for both graphics and data processing. Clayton also provides coverage of the Metal Compute Pipeline, demonstrating practical GPU programming applications ranging from image processing to neural networking.
- Quickly get a basic Metal project running
- Work with Metal resources and memory management
- Learn how shaders are compiled and accessed by the CPU
- Program both 2D and 3D graphics with Metal
- Import 3D models and assets from Blender, Maya, and other programs
- Apply imported textures to model objects
- Use multipass rendering to efficiently implement computationally expensive techniques
- Leverage tessellation to reduce mesh detail
- Use the GPU for a wide spectrum of general-purpose computing applications
- Get started with the Metal Performance Shaders Framework
Preface xvii
Acknowledgments xxiii
About the Author xxv
Part I: Metal Basics 1
Chapter 1: What Is Metal? 3
History of Graphics APIs 4
Metal: The New Way to Do Graphics on Apple Platforms 6
Metal in Context: How Metal Complements and Supports Other Platform Frameworks 10
Summary 10
Chapter 2: Overview of Rendering and Raster Graphics 13
Representing the GPU 14
Preparing Data for the GPU 16
Summary 24
Chapter 3: Your First Metal Application (Hello, Triangle!) 25
Creating a Metal Application in Xcode (without Using a Template) 25
Creating a MTLDevice 28
Creating a CAMetalLayer 28
Creating a Vertex Buffer 29
A First Look at Shaders 31
Libraries, Functions, and Pipeline States 34
Introducing Render Passes 35
Introducing MetalKit Features and MTKView 39
Summary 40
Part II: Rendering and Graphics 41
Chapter 4: Essential Mathematics for Graphics 43
Language of Mathematics 43
Coordinate Spaces and Moving among Them 44
Points, Vectors, and Vector Operations 46
Normalization and Unit Vectors 49
Pythagorean Theorem 50
Sine, Cosine, and Tangent 52
Matrices and Matrix Operations 53
Transformations: Scale, Translation, Rotation, Projection 55
Summary 60
Chapter 5: Introduction to Shaders 61
Metal Shading Language Overview 61
Setting Up Shaders 63
Your First Shader: Pass Through 63
Writing Your First Shader 68
Uniform Buffer 74
Summary 82
Chapter 6: Metal Resources and Memory Management 83
Introduction to Resources in Metal 83
The Argument Table: Mapping between Shader Parameters and Resources 84
Buffers 85
Resource Options: Storage Mode, Cache Mode, Purgeability 86
Preparing Data for the Vertex Shader and Vertex Descriptors 87
Copying to and from Buffers 88
Introduction to Textures 89
Copying to and from Textures 93
Compressed Texture Support 94
The Blit Command Encoder 94
Generating Mipmaps 96
Summary 97
Chapter 7: Libraries, Functions, and Pipeline States 99
What Are Libraries and Functions? 99
The Metal Two-Phase Compilation Architecture 100
Creating Libraries at Compile Time and Runtime 101
Command Encoders 103
Render Pipeline Descriptors and State 104
Pipeline Reflection 105
Summary 105
Chapter 8: 2D Drawing 107
Metal Graphics Rendering Pipeline 107
Sample Project: Build a Star 108
Metal Primitive Types 113
Responding to MTKViewDelegate Methods 115
Retrieving a Drawable 115
Creating a Command Buffer 116
Creating a Command Encoder 117
Fixed-Function State on the Command Encoder 119
Passing Data to Shaders 121
Issuing Draw Calls 124
Scheduling and Enqueuing Command Buffers 124
Summary 126
Chapter 9: Introduction to 3D Drawing 127
Model-View-Projection Transformations 127
Clip Space and the View Frustum 131
Shading Models 136
Basic Lighting 138
Animation 140
Summary 141
Chapter 10: Advanced 3D Drawing 143
Constructing a Hierarchical World with Scene Graphs 143
Instanced Rendering 145
Summary 155
Chapter 11: Interfacing with Model I/O 157
What Are Model Files? 158
Importing a Model 161
Meshes and Submeshes 163
Render State Pipeline 164
Asset Initialization 165
Render State Setup and Drawing 166
Exporting Files 167
Summary 168
Chapter 12: Texturing and Sampling 169
Texture Mapping 169
Mipmapping 171
Sampling 172
Precompiled Sampler States 175
Passing Textures and Samplers to Graphics Functions 177
Summary 182
Chapter 13: Multipass Rendering Techniques 183
When (and Why) to Use Multipass Rendering 183
Metal Render Pass Descriptors 184
Creating and Managing Render Targets 185
Revisiting Load-Store Actions 188
Summary 190
Chapter 14: Geometry Unleashed: Tessellation in Metal 191
Removing Bottlenecks by Using Tessellation 191
Catmull-Clark Subdivision 192
Per-Patch Tessellation Factors 195
Metal Tessellation Fixed-Function Pipeline 197
Setting Up a Tessellation Kernel 201
Post-Tessellation Vertex Function 202
Draw Patches 204
Summary 208
Part III: Data Parallel Programming 209
Chapter 15: The Metal Compute Pipeline 211
Introduction to GPU Programming 212
Concurrency versus Parallelism 213
Using GPUs for General Computation 216
Kernel Functions 217
The Metal Compute Command Encoder 218
Issuing Grids of Work 220
Finding Your Way in the Grid inside the Kernel Function 223
Reading and Writing Resources in Kernel Functions 223
Summary 224
Chapter 16: Image Processing in Metal 225
Introduction to Image Processing 225
Creating a Metal Texture 227
Desaturation Kernels 230
Convolution and Dispatching a 2D Grid 232
Blur Effects 235
Selecting an Optimal Threadgroup Size 238
Summary 239
Chapter 17: Machine Vision 241
How a Computer Sees the World 241
Noise and Smoothing 242
Sobel Edge Detection 244
Thresholding 245
Histograms 246
Facial Recognition 246
Summary 253
Chapter 18: Metal Performance Shaders Framework 255
Overview of Metal Performance Shaders Framework 255
Image Processing with the MPS Framework 257
Matrix Operations with MPS 267
Summary 269
Chapter 19: Neural Network Concepts 271
Overview of Neural Networks 271
Neural Network Components 272
Neural Network Architecture 281
Summary 283
Chapter 20: Convolutional Neural Networks 285
History of Convolutional Neural Networks 285
MPSImage 289
Convolutional Neural Network Kernels 290
Convolution Data Source 296
Neural Network Graph 298
Summary 299
Index 301