This is a post I’ve wanted to put together for a while now, but I haven’t exactly known what to call it or describe what it is I’m trying to get across. But with the flurry of activity I’ve seen on twitter recently surrounding the Mixed Reality stuff coming in the Windows 10 Fall Creators Update on October 17 , I thought it was time to cut my losses and just put pen to paper.

I’m a dev, and for the last 10 or so years I’ve been writing code professionally on various different platforms but most of it has been done on Windows and I’ve tended to lean towards being a front end guy. I’ve been doing XAML/C# through all the different flavours from Avalon up to UWP/Xamarin and (I like to think) I’ve gotten pretty good at writing apps that, fundamentally, crunch data and present the results on screen for users.

Doing this, I’ve learnt a ton of other stuff that can be applied to other things such as design patterns, performance tweaks, what tools can be used to get a job done and the list goes on.

If you are reading this, you are more than likely a developer in this position, and probably way smarter than me and way better at it then I can ever hope to be.

Also, like me, you would probably watch one of the recent Mixed Reality videos, such as this one…

..and think WTF. How do I do that?

and this gets a little bit more interesting when you see slides such as this one from the recent Microsoft Ignite conference.

That’s Satya Nadella explaining where Microsoft is placing its bets over the coming years - In Mixed Reality, Artificial Intelligence & Quantum Computing.

What’s the point I’m trying to make? Well, I think there are going to be a lot of developers like us who will want to perhaps shift focus and learn some of this stuff seeing as the company is making a big shout about it and, as it turns out, I can possibly help with the Mixed Reality part.

I spent the last year and a half working at Microsoft on the Paint 3D team, and I was able to get about a years’ worth of 3D graphics and gaming knowledge from the devs, most of which had previously worked on games such as Crysis, Quantum Leap, Starfox and a ton of others.

Obviously, this wasn’t just handed to me on a plate, as we were all too busy getting Paint 3D off the ground. But on the side I did learn (and I am still learning!) about this other world of 3D graphics.

I was very disappointed with the resources that are out there with regards to this. If I wanted to learn, say, JavaScript or even a package such as Photoshop I can run over to places like Pluralsight and within minutes I can be watching a 3-7 hour course to hit the ground running. With 3D graphics on windows I found very little, and the stuff I did find was either out of date or not relevant anymore.

What I want to present over the next few posts (or, I may just update this one) are few technical & non-technical resources That I have used and am still using to get better at this.

DirectX

If you stop reading now and run off to google “getting started with Windows Mixed Reality development” you will very quickly run into the Unity game engine which is what is primarily used to create 3D experiences on these devices.

I’ll get on to 3D engines in the next update, but I’m a “bottom up” kinda guy so I want to start right at the bottom as close to the metal as possible, and that starts with DirectX.

DirectX is a broad name given to a ton of Windows APIs that are geared towards multimedia and video applications which is currently on version 12. There is a sub section of DirectX, known as Direct3D, that is used to render three-dimensional graphics in applications such as games.

I’ve found that by learning how DirectX works and theory behind it has helped me leaps and bounds when I’ve moved up the chain to more high level platforms such as Unity - but this is obviously the long haul approach and might not be suitable for everybody.

To start right at the beginning, I highly recommend watching this video by @ChasBoyd which was given at an Intel conference and quite literally begins with “folks like intel take sand and turn it into silicone worth $500+”

Shaders

As you start exploring the DirectX API, It won’t take you long to run into Shaders. Think of these as small, separate programs that take in pixel data, manipulate it in some way by adding an effect for example, and then return the manipulated data back to the caller. These have their own language (HLSL - High Level Shader Language) and are applied to the 3D scene later on in the pipeline.

Books

There are two books that I used to start my DirectX journey. The first I’ll just go ahead and say is pretty much essential reading and the second gave me a bit more knowledge but some of the tools are now slightly outdated.

3D Game Programming with DirectX 12 - Frank D. Luna

Amazon Link: Here - Note that this is currently only showing up as a Kindle eBook. You may want to have a hunt around for a print copy.

At close to £30 this book isn’t cheap but IMHO it’s completely worth it. Luna’s writing style, while very methodical, is thorough and easy enough to understand.

The book starts out right at the very beginning without a computer screen and takes you through the maths that are required to understand how to compute in 3D - Vectors, Matrices and Transformations. Back to pen, paper and a calculator!

The book then moves on to discuss how to initialise D3D, how to render, draw, create lighting, how to use Shaders, Mapping, the works! There is even an appendix on Windows programming in C++ if you are a little rusty on that.

Because this is so low level, you are literally starting with a blank screen. Luna has taken the liberty of creating a small, basic 3D engine that is used through most of the examples, and the latest version of the code on his GitHub page will quite happily work with Visual Studio 2017 on Windows 10.

There are one or two flaws, however. I did find that there were pages and pages of source code with very little description into what it was doing, so I was googling words and reading up on the MSDN documentation. I’m currently on my second read through of this and I have it as a reference to look at all the time.

Real-Time 3D Rendering with DirectX and HLSL - Paul Varcholik

Amazon Link: Here - Note that this is also showing up as a Kindle eBook, and the physical copy is close to £400! Might want to shop around for this ;)

This book I didn’t quite get to the end of, but I did find it helpful in explaining Shaders and how to author them. Again, the book suffers from having long paragraphs of code with little explanation, so you may find yourself googling words and reading extra documentation.

The book also realises heavily on the NVIDIA FX Composer tool which is slightly old now and hasn’t been updated since Windows 8.1.

Tools

Here are some of the tools I’ve dug out and use for things related to 3D development. To try and keep this short I’ve left out the obvious thing’s such as Visual Studio.

ShaderToy

Price: Free

ShaderToy is an online community for creating and sharing shaders in the browser. Users can submit shaders which render in the browser and you get full access to the source code.

ShaderTool

Price: £22.99 on Steam

ShaderTool is a desktop based IDE for shader development and it’s pretty awesome. You will need a Steam account to buy it as it’s not currently available by any other means. This would be a prime app to add to the Windows Store so maybe that will happen.

Nvidia FX Composer

Price: Free

FX Composer is another shader IDE. This one is free but the downside is it’s quite old and hasn’t been updated since Windows 8.1. Some hacks are required to the project templates to work with the latest version of DirectX and these are described in Varcholik’s book described earlier.

Microsoft Mathematics

Price: Free

This free calculator tool is great. It’s a little old and was primarily aimed at high school students but it’s a great visual tool for working out plots in 2D and 3D & step-by-step equation solving.

Blender

Price: Free

Blender is a free and open source 3D creation suite for creating, animating and rendering 3D models. While the industry tend to use paid up versions such as Maya and 3DS this is a great, free, starting point.

Links

C++ for C# Devs - A Somewhat Short Guide I found as a great resource for brushing up on C++ syntax using what I already know as a C# developer.

DirectXTK - The DirectX Toolkit is a great library for getting basic functionality into your app/game. Things like loading 3D models, working with textures etc are all provided here.