Windows 10 Creators Update & Paint 3D

Windows 10 Creators Update

With the Windows 10 Creators Update RTM shipping recently (and the next update being revealed at Build 2017) I thought it would be fun to share some of my experiences working with the Paint 3D team. This won’t be a technical deep dive into how the app works it’s purely me thinking over what went on over the last 2 years and jotting down anything that springs to mind. Technical details are boring anyway.

I joined Lift London in October 2015 as a freelance engineer. Lift are (or were) a game studio who, at the time, sat within the Xbox division at Microsoft and were primarily concerned with making video games.

This meant that they were essentially a remote office that sat within the organisation structure of Microsoft Corp in Redmond, Washington and not Microsoft UK.

At the interview they couldn’t tell me what they were working on, but what they could say was that they were working on a Windows Store App that involved 3D & DirectX and that they were looking for an engineer to focus on the app specific parts of the project - enter yours truly.

Being a dev who has focused on app development, it was clear that while these super smart guys had experience in astrophysics, simulation, animation, texturing and other complicated things, they knew how to make games which is a different ball game to an app.

I love video games and while i’ve never actually worked on one (come close a few times), I do have a bit of knowledge on how they hang together, so this was a perfect opportunity for me to offer up my app experience to the team while at the same time learning about the world of 3D graphics, DirectX and Hololens direct from the source.

This really is as cool as it sounds, if you are a gamer that is. I got to chat to engineers & executives in a bunch of different studios, hear about and try new hardware and software years away from release and beta test the latest builds of their projects. We had consoles set up to do this and it was actively encouraged that we did do that. Lunch times playing Recore and Forza spring to mind.

We also made some awesome videos over the year that became I kind of interstudio competition to see who could make the best one. Here is our Mannequin Challenge vid from last year.

Paint 3D

When we started the app wasn’t known as Paint 3D we were just simply calling it Paint, and by the time I started we had a rough idea on what we wanted to build but a lot of how it was going to work hadn’t been thought through just yet. The way the app works is a new paradigm that hasn’t been done before. As you can tell from the end product, this isn’t just a receive json, bubble up to UI, edit and save affair like most apps are today.

The app was also a very large puzzle piece in a much bigger 3D story with the other pieces being created by other studios around the world. Paint became the “creation” piece that would take centre stage in the “3D for Everyone” program.

Engineering

On the engineering side their were two teams, Which we called Prototype and Production. The goal was simple - the Prototype team would steam on ahead and try out ideas, test them, take on feedback from the studio and test users, refine them and then come up with the model on how the app was going to work, and the the Production team would be a few steps behind writing what would eventually become the completed app in production code.

Prototype were using UWP along with Unity 3D (in C#) to get their idea out quickly, while Production were targeting UWP with C++/CX.

The combined engineering team worked alongside a design and UX team who did exactly that - designed the app from a UX perspective, provided all of the assets etc.

Prototype

Having this working prototype was extremely valuable not just as a test for our products direction but also as a “Show me” piece for Microsoft at large. Microsoft is an extremely large corporation made up of dozens of teams and product groups so when we had to engage with one of these teams, having a working prototype on what were trying to do was gold and built up alot of excitement within the organisation.

Building Paint

To build Paint we adopted a process using sprints to deliver a shippable product every few weeks. We would review each others code and demonstrate new capabilities as we were working on them not just to the studio but also to the executive team both remotely and on the various studio visits they did throughout the year.

As the sprints rolled along, the app we were creating was really cool but it was really hard to visualise how folks (especially kids) were going to use it. So we took the app to a local US school and got them to use it, and we made a video that was used internally at first but was then used as part of the announcement. When I saw this, this was the first real aha! moment and it really did make us focus on making a decent app.

Paint 3D: Bring your ideas to life in 3D

Something quite cool about this project was just how much our product was being talked about with the grown ups at Microsoft Corp. It wasn’t uncommon for us to get some feedback from folks well known even outside technology world.

One such case involved a few of the guys who created technologies we used to create Paint, and I remember reading the text books written by these guys late into the night in my younger university years.

Having them visit us from the states and having them sit with us while we were explaining how we were using the tech they invented, usually in ways they didn’t even imagine it would be used, was definitely a full circle moment for me.

Microsoft October 2016 Briefing

by Mid 2016 we had the app in a good place and an Alpha version was released internally. Once this was at a decent release quality we decided that the next deadline would be the one where we went public, and the app wound be announced at the October 2016 briefing along with the Surface Studio.

By this point the prototype had done it’s job and we were now one development team focusing on Paint in Production (and we still had another team working on the lower level “Viewer” technology which is a component that Paint uses).

We really were working on the app up until the last minute, I remember being in the office at 12AM writing the code that became the EULA the night before launch.

Then, on the 26th October at around 10am in New York, I was in the London office with a beer in hand to watch our team leader Megan get on stage and announce Paint 3D to the technology journalists of the world.

Paint 3D was to become the main piece in the “Creators Update” which was coming to all Windows 10 users in the Spring of 2017.

Introducing the Windows 10 Creators Update

The feedback was pretty awesome. A lot of it was, of course, focused on the 3D elements of the app but I remember reading a few articles that praised the fresh UI and also the level of innovation that folks were not used to seeing come out of Microsoft.

Road to RTM

After the launch at the press event, we now had live users using the app providing feedback and feature suggestions. I’m not even kidding when I say that every single piece is taken on board, discussed, prioritised and then added to the workload for a given sprint. Feedback from users matters.

our next real deadline became the Creators Update ship date and the version of Paint 3D we were working on would become part of the Windows Image that millions would install.

Bit of a nerd bit here, but as we were part of the Windows division, and Paint was part of the Windows Image, we had full access to the Windows source code. Not only that, but you could set up Visual Studio to debug directly into the Windows source

Day 1 Update

What’s interesting with the RTM release date is that we actually had to have the “Final” version of Paint 3D ready for the image several months before the Creators Update released to the public. This is because it has to go through several rounds of testing, particularly with Windows Insiders, before the image can go public.

When we passed this milestone, we worked on a “Day 1 Update”. This version of the app would be pushed to anybody who had installed the Creators Update with an internet connection and would be pulled from the Windows Store during setup.

It’s not very often you get to work on a baked in App so it was interesting to go through the notions of managing all the different versions that were required.

In this time, my development work shifted to working on new Windows UI things that weren’t just Paint specific, but were also to be used across Windows. This set of design patterns got announced at Build 2017 as the Fluent Design System and elements of this exist in Paint 3D today.

Ship it!

With Paint 3D and the Creators Update shipped my time at Microsoft was done (at least this time around) It’s easy to say that the guys you get to work with on a project are “best in the industry” and a ton of other rubbish buzzwords but these guys were the real deal. This is probably the most fun I have ever had on a project and I learnt so much in 18 months I felt physically drained by the end of it.

No doubt I’ll be back for the next big adventure