There's one other place where fog snaps right now, and that is in time -- it is probably pretty easy to fix, though the explanation might be a little unexpected so I am Ccing it to the team as a general knowledge transfer thing.

In general, with floating point numbers, there is a problem with loss of precision when you do computations involving large numbers added to small numbers; the small numbers tend to get dropped and go toward zero. This is a general thing about computers, not just graphics (it is the price we pay for being able to pretend we can store real numbers in small amounts of memory). Most of the time it's not an issue and programmers don't think about it, but there are some times when it can be an issue.

In shaders, we often want to have a time variable, controlled by the CPU, that we can use to help generate effects. That time variable counts upward, maybe from the time the level started, or the time the game started. If that number is measured in seconds, then after 10 minutes it will be around 600, after 2 hours it will be 7200, etc. If someone leaves the game running for 3 days (a certification requirement on consoles!), it would get up over 250,000.

Those numbers don't seem super big, but at the same time, we also deal with numbers that are small. If the game is running at 60fps (which on PC these days is considered maybe a slow frame rate!), that's 0.017 seconds per frame, and we might be doing math that involves both this dt and the current time. At which point the ratio of the current time to the frame time is like 250,000 / (1/60) = 15 million. That is really very high and a recipe for all kinds of numerical problems you don't expect -- if you add the delta time to the current time, in a 32-bit float, the delta time is almost completely destroyed because there is not enough precision in the number. Long before this time, you'd start to see jitteriness in animation, things not matching up to where you'd expect them to be, etc.

One way to solve this is not to let the current time grow indefinitely -- at some point, you reset it to 0. You could pick a fixed amount of time, like I don't know, every minute, and at the end of the minute the time is back at 0 and you keep going. But if you want your effects to be seamless, they all have to match up exactly at the 1-minute boundary, which makes tuning things difficult and annoying. For example, if you want to make a cosine wave that is seamless across time, you can make the wave with a period of 1 minute, or half a minute, or 1/3 a minute, etc, and firstly this is annoying because when designers want to tune stuff, they have to know this, and it's unwieldy and confusing, and secondly, these numbers cannot be represented exactly in floating-point numbers either, so you're adding imprecision that way.

But there is an old graphics programmer trick that solves this very cleanly, that I think has been around for a long time (I first heard of it from Ignacio when we were working on The Witness).

For our cosine wave example mentioned above, we are going to be evaluating some function like cos(freq*time*2*PI), where 'time' is just the wall clock time according to the game, and 'freq' is some parameter controlling the frequency of the cosine wave, that we want it to be very easy for designers to adjust without introducing problems.

The constraint you have to meet, for your cosine waves to match up when the timer resets, is that they all have to evaluate to 1 at whatever the max time is -- because when you reset to time = 0, you will get cos(freq*0*2*PI), which is 1. So cos(freq*time_max*2*PI) also has to be 1, in other words, the cosine has to have gone around the circle an integer number of times. Since we have a factor of 2*PI in there, this means that our wave will be flawless any time freq*time_max is an integer.

How do we ensure that, easily, in a way that people don't have to think about? Pick, for time_max, a number of seconds that is 10 to some integer power -- for example, 1000. Then, any number you type into a tweak file, that has no more than 3 digits after the decimal, will result in an integer number of times around the circle. For example, if freq = 9.876, we get cos(9.876*1000*2*PI) == cos(9876*2*PI) == 1. This works for any number you pick with 3 digits after the decimal. If you need 4 digits, make time_max 10000, and so forth. If we are reading values from a tweak file, we can round them to the nearest 3 digits before sending them to the shader (and maybe output a warning, that the digits aren't being used, if the rounded number is appreciably different from the input number, to remind people of this system).

This works with any periodic function, we just used cosine here as an example. For linear shifts, like scrolling uv coordinates or whatnot, it's similar -- if you have uv coordinates from 0 to 1, you just need to make sure shift_rate*time_max is an integer in order for the scrolling to be perfect when the time resets. Similarly for the time parameter used to index a flipbook animation.

So the reason the fog is snapping after you play for a while is that the time sent to the shader system gets reset to 0 every 1000 seconds but the fog doesn't know about this. And the reason for the resetting is to make sure we maintain good precision in the time-based effects in the shaders.

Congratulations to the following creators:

Anastasia Kuznetsova - Raison
AWK - Harmon:i:c
Dario Zubović - QBCO
29Games - RocketPuzzle
Asriele Aestas - Our Sky
Cyathea Tree Team - Out of Bounds Exception
David Rhee - Sticky Blocks
Nico Saraintaris - Trash Diamonds
Weird Sisters Interactive - It's a Zoo.
Ruby - SokoBunny
Skalmantas Šimėnas - Magnecopter
Andrius Mitkus - Rimtis (working title)
Jose Hernandez - Efficient Necromancy

We are excited to see how these games progress and we're looking forward to sharing updates.

---

Jon wanted to convert his demos into proper language tests, and add in new ones. Testing becomes increasingly important as we approach a closed-group release. As it turns out, the literature on compiler testing is scarce, and getting access to quality test suites is pricey (see C or FORTRAN). I've also disliked test frameworks in general, which added to the problem.

In the interest of feeling joyful and productive, I decided to sit and think about the specific reasons why I've been unhappy with frameworks, and to use the language to write one I'd enjoy using.

How I Like to Test

I like to ask personally interesting questions which the tests ought to help me answer. Often, they relate to the behavior of software at the threshold of cross-cutting concerns. For the jai compiler, they could be questions such as:

• Can we make a procedure's return type randomly change at compile-time?
• What if a polymorphic procedure is used as a constructor?
• How do we prevent certain structs from using a procedure as a constructor?
• Can we generate bytecode for a slightly different procedure because of the struct that is using it as a constructor?

Note: We tend to use the term procedure instead of function.

Don't worry if the questions seem bizarre (the language is new and hasn't been released yet). The common theme is they all require very different language features to interact with each other (i.e. cross-cutting). For example, the first item could be handled polymorphically with the modify directive, enums, a switch statement, and some random bit generator. The answer to the second item is that nothing bad happens; in fact, this opens the door for multiple structs to use the same polymorphic procedure as their constructor. This observation naturally leads to questions three and four. When I wrote the tests for some of these, compiler bugs were uncovered naturally and without much effort.

Note how these questions are answered by just writing programs. They are the language tests. Anything not helping me towards that feels like noise, and that explains why I haven't been happy with many testing systems. Here's the criteria I want a framework to meet to feel good about writing tests:

1. Ability to write tests as if they were real, independent programs.
2. Simple asserts the tests can use.
3. By default, virtually zero interaction with the test framework.
4. Minimal test suite reports that are nice to look at.

By a minimal report I mean showing me which programs ran and when, how many test asserts were called and where to find them if they failed. The report should look good so I can help my brain enjoy the overall experience.

Our Framework

To showcase the framework we ended up creating, let's go through an example, transforming a program into a language test.

Here's the program that tackles one of the simpler inquiries from earlier: What if a polymorphic procedure is used as a constructor? While we're at it we'll ask the same of destructors.

#import "Basic";

Thing :: struct {
    mem : *u8;
    value := 42;
    #constructor init_thing;
    #destructor free_thing;
}

init_thing :: (using thing : *$T) {
    mem = alloc(1000);
}

free_thing :: (using thing: *$T) {
    free(mem);
    mem = null;
    print("Thing memory freed.\n");
}

main :: () {
    {
        our_thing : Thing; // Constructor fires off.
        print("%\n", our_thing);
    } // Destructor fires off.
}

If we compile it, we see that it's valid:

 >>> jai poly_constructor.jai
 >>> poly_constructor.exe
 Thing { value = 42; mem = 1e4e6b4f790; }
 Thing memory freed.
 >>>

As I had mentioned, this means multiple structs can use the procedure as their respective constructors.

 #import "Basic";

Thing :: struct {
    mem : *u8;
    value := 42;
    #constructor init_thing;
    #destructor free_thing;
}

AnotherThing :: struct {
    mem : *u8;
    message := "I don't hold the meaning of life.";
    #constructor init_thing;
    #destructor free_thing;
}

init_thing :: (using thing : *$T) {
    mem = alloc(1000);
}

free_thing :: (using thing: *$T) {
    free(mem);
    mem = null;
    print("Thing memory freed.\n");
}

main :: () {
    //
    // Test with multiple structs.
    //
    {
        our_thing : Thing;
        print("%\n", our_thing);

        different_thing : AnotherThing;
        print("%\n", different_thing);
    }
}

Output:

 >>> jai poly_constructor.jai
 >>> poly_constructor.exe
 Thing { mem = 1d7df5bf790; value = 42; }
 AnotherThing { mem = 1d7df5c0190; message = "I don't hold the meaning of life."; }
 Thing memory freed.
 Thing memory freed.
 >>>

Great! And remember: this is just a program. We can explore the possibility space even more, define any number of procedures, declare hundreds of complex structs, import 3rd-party modules—you name it. However, let's pretend we're done answering our polymorphic constructor questions. To turn this into a language test, let's make two straightforward modifications (we'll add a couple of asserts while we're at it):

polymorphic_constructor :: () {
    //
    // Test multiple things.
    //
    {
        our_thing : Thing; // Constructor fires off.
        assert(our_thing.mem, "Thing didn't get an memory address.");

        different_thing : AnotherThing;
        assert(different_thing.mem, "AnotherThing didn't get an memory address.");
     }
 } @TestProcedure

Notice I changed the name from main to something else and added the simple note TestProcedure.
Notes are essentially comments recognized by the compiler that do nothing by default.

Let's put that file in a tests/ folder.

 >>> mv poly_constructor.jai tests/
 >>> ls
 tests/
 tests.jai
 >>>

Ah, the reader will notice I have a file conveniently called tests.jai. Let's compile it.

 >>> jai tests.jai
 >>> ls
 tests/
 tests.exe
 tests.jai
 >>>

That produced tests.exe; let's run it! We'll use some real screenshots this time.

And we just made a full test suite run.

Note: The reader might ask why we have 82 more asserts than what we wrote, plus another test procedure
from a String file. More on that soon.

Notice a log file was generated. Here's the summary portion of that report:

Report format is based off Aras Pranckevicius' API doc

That's a lot of useful output just by compiling a source file! In addition, we can have multiple test procedures on the same test file:

    polymorphic_constructor :: () {
        //
        // Test multiple things.
        //
        {
            our_thing : Thing; // Constructor fires off.
            assert(our_thing.mem != null, "Thing points to null.");

            different_thing : AnotherThing;
            assert(different_thing.mem != null, "AnotherThing points to null.");
        }
    } @TestProcedure

    polymorphic_constructor_uninit :: () {
        //
        // Control when the initializer and constructors
        // actually get called.
        //
        {
            our_thing : Thing = ---; // Uninitialized var.
            memset(*our_thing, 0, size_of(Thing));

            T :: type_of(our_thing);
            tis := cast(*Type_Info_Struct) type_info(Thing);
            initializer := cast((*T) -> void) tis.initializer;
            constructor := cast((*T) -> void) tis.constructor;

            initializer(*our_thing);

            assert(our_thing.value == 42, "Expected % but got %\n", 42, our_thing.value);
            assert(our_thing.mem == null, "Expected % but got %\n", null, our_thing.mem);

            constructor(*our_thing);

            assert(our_thing.value == 42, "Expected % but got %\n", 42, our_thing.value);
            assert(our_thing.mem != null, "Didn't want % but got % anyway", null, our_thing.mem);

            //
            // We can do the same for AnotherThing.
            //

            // ...
        }
} @TestProcedure

Opening the report we get:

If we wanted to, we could drop in more test files in the tests/ folder, each containing their own sub-programs. Finally, if invariances in our tests don't hold (i.e. an assert fails), that will also be reported. Let's add a deliberate failure on the first test procedure:

    polymorphic_constructor :: () {
        //
        // Test multiple things.
        //
        {
            our_thing : Thing; // Constructor fires off.
            assert(our_thing.mem != null, "Thing points to null.");

            //
            // Deliberate failure.
            //
            assert(our_thing.value == 999, "Meaning of life isn't 999? It's %", our_thing.value);

             different_thing : AnotherThing;
             assert(different_thing.mem != null, "AnotherThing points to null.");
         }
} @TestProcedure

And we get that on the log file too:

If the test writer heads to the test files section, the procedure that failed would be made apparent.

Library Tests

So how do we explain that weird String file from the reports earlier? We didn't write that! Well, Ignacio found it useful to use this framework for the language's system modules. System modules are special .jai files we provide programmers. Users can import them to leverage OpenGL / D3D, font rendering, audio playback, and more.

The test file from earlier imported Basic (a catch-all module we use to experiment new features). Basic
happens to indirectly import the String module. When I opened up String, I noticed Ignacio had written this somewhere on the file:

string_tests :: () {

    assert(join(..{:string: "foo", "bar", "puf"}, ", ") == "foo, bar, puf");
    assert(join("foo", "bar", "puf", separator="/") == "foo/bar/puf");

    assert("foo/bar/puf" == join(..split("foo/bar/puf", "/"), "/"));
    assert("foo\\bar\\puf" == join(..split("foo/bar/puf", "/"), "\\"));
    assert("foo/bar/puf/grr" == join(..split("foo, bar, puf, grr", ", "), "/"));

    // More asserts
    // ...
} @TestProcedure

Which accounted for the 82 additional asserts. That's because the framework detects tests written in external modules, outside of our own test files. Now anyone on the team writing modules can write test procedures inside their module, tag them, and they're done. In fact, I have a file under tests/ that just imports all the modules. The actual tests can live right next to the module code, outside of our folder, and we register them for free just as we automatically register the test files inside the folder for free. And fret not, a module being imported multiple times will still have its test recognized only once.

Future Updates

I'd like to note there are special settings and configurations the user doesn't know about (good! that's the point!), and those who care can open up the build file, which is just the tests.jai source file. Documentation amounts to a few well-placed comments.

Upon release (if we have time to make it happen), I'd like people to write their numerous test procedures, compile tests.jai, and have them witness Jon's bananas.jai but in the context of tests: A visualization of a growing tree of test files as branches, where each test procedure is a leaf from the branch.

How It All Works

A common theme back at Kennedy Space Center was that we wanted to automate most of the "bookkeeping" so the engineer's time and effort went into thinking about the real problem. Interacting with build systems was not in our best interest! Even so, sometimes we were stuck with what I think were high-friction tools. I can't properly express to the reader how great it's been to write a tool that reduces friction nontrivially.

Let's take a quick look at how the language played a part in letting me create this framework, as I think it has the right kind of features all statically-typed languages should have going forward.

Minimizing Framework Interaction

Initially, the main struggles in making the testing system work were psychological, working to unlearn years of how I was told automated test frameworks should be structured. My first attempt was writing a hybrid C++/Jai program with wacky command-line parameters. I think Jon nearly had a heart attack. It was difficult to implement something that met the criteria listed at the beginning of this post, because my model was tied to old-school software testing models.

The closest implementation I have ever found is CxxTest. What caught my interest was the second sentence on their homepage:

CxxTest is easy to use because it does not require precompiling a CxxTest testing library, it employs no advanced features of C++ (e.g. RTTI) and it supports a very flexible form of test discovery.

Aha! It's trying to make itself invisible, and the term test discovery sounds like it's doing work on behalf of the user. Indeed, later the website goes on to say:

Additionally, CxxTest supports test discovery. Tests are defined in C++ header files, which are parsed by CxxTest to automatically generate a test runner. Thus, CxxTest is somewhat easier to use than alternative C++ testing frameworks, since you do not need to register tests.

Aha por dos! Test discovery is the property I want frameworks to exhibit. With CxxTest, though, we still need to understand its build system steps to reap the benefits, but that's a difficult limitation to overcome with C++. In jai, however, we can specify our build system in the language.

So I made a source file, tests.jai. In said file, we can ask the compiler to compile any number of different files, and request it to open up its compilation stages to us as things get lexically scanned, parsed, and code generated. That's more than we care to ask for, so our entire build system can amount to compiling a single file, in the same way we compile any normal code.

Tagging Procedures

As I just said, in tests.jai we can ask the compiler to pause at any compilation stage for us to intervene in the process. So I used that for the test discovery. The current mechanism, as we have seen, is to tag procedures that want to behave like an independent test program with the note @TestProcedure. Then, once the compiler has finished type-checking all source files, I ask it to give me the declarations tagged with the note. Specifically:

for decl.notes {
    if it.text == {
        case "TestProcedure";
        proc_info := cast(*Type_Info_Procedure) type;
        if proc_info.argument_types.count > 0 {
            print("[Test Suite] WARNING: '%' tagged with @TestProcedure should take no arguments.\n", proc.name);
            continue;
        }
        user_test : TestProcedure;
        user_test.filename = decl.filename;
        user_test.procname = proc.name;
        array_add(*tests_to_register, user_test);

        //...
        case;
        // Do nothing.
    }
}

Where decl is the current code declaration we're inspecting from the AST.

In the past, I used to check for procedure names starting with test, but Allen and Nico made me question whether that was too constraining. Turns out it was. The downside with the notes here, though, is there might be tests importing external modules that tagged some of their code with @TestProcedure for unrelated reasons. I mitigate that by ignoring structs with that tag, and by verifying whether the procedure acts as an entry point (i.e. main with no parameters). There are other options available to us, but for now this is a good-enough solution.

Hidden Test Suite

I register the discovered tests by creating register_test procedure calls on-the-fly--meaning I update the program's AST before things enter the bytecode pipeline. Where is the register_test procedure defined? Well, the dirty secret is I still wrote an actual test suite, contained in a single file living inside the tests/ folder. The build file has access to that, as do people. If anyone wishes to avoid all the automatic magic and also believe editing the build file is not enough, they are free to ignore it altogether and figure out how the suite works. They can register tests manually, generate reports however they see fit, and perform the entire job of a framework themselves.

Test Asserts

As we can see from the CxxTest homepage, it has different types of asserts for us to use. I had something similar going, but Ignacio reminded me that Jon added an assertion handler as part of the implicit context. That means we can override the behavior of regular asserts! This is what allows external modules to write officially-recognized tests inside their file and still compile anywhere--we're not requiring them to do anything special.

What's Next

I now enjoy writing tests whenever I need to, and I hope this straightforward style sounds appealing to some people.

In other news, for the purpose of testing a compiler, will an automated test suite suffice? Not really, though it's a necessary first step. Although we've uncovered dozens upon dozens of important compiler bugs, we're starting to hit the ceiling of what human-made tests can offer. It's no surprise we've started to seriously look into a different beast entirely—a compiler fuzzer.

But that's a story for another blog post!

P.S. Shoutout to Josh, who among other things, helped write a bunch of the language tests and makes sure everything keeps running smoothly on Mac and Linux.

You can find it on the App Store, and carry around the full game in your pocket...which is pretty crazy if you ask me.

The game will work with iOS 10 and up, on:
iPhone 5S (or newer)
iPad Air (or newer)
iPad Mini 2 (or newer)
iPod Touch 6th Generation

Oh, also, it's only $9.99!

The Witness requires Metal for rendering. Metal is a graphics technology from Apple that allows faster and more fluid graphics. These are the Mac computers that support Metal:

• MacBook (Early 2015)
• MacBook Air (Mid 2012 or newer)
• MacBook Pro (Mid 2012 or newer)
• Mac mini (Late 2012 or newer)
• iMac (Late 2012 or newer)
• Mac Pro (Late 2013 or newer)

https://support.apple.com/en-us/HT205073

It's possible to build custom Mac systems with Metal support, either using an old Mac Pro and upgrading the graphics card or building it from scratch using compatible parts (Hackintosh). These configurations are unsupported, but some customers have found them to work.

If using an AMD GPU, the only supported add-in card with Metal support is:

AMD Radeon 7950 (Tahiti Pro) Sapphire card.

If using an NVIDIA GPU, any GeForce 6xx or newer should work, but it is necessary to install the latest drivers from:

http://www.nvidia.com/object/mac-driver-archive.html

System Requirements:

Minimum:

* OS: El Capitan 10.11.6
* Processor: Intel Core i5 @ 1.8 GHz
* Memory: 4 GB RAM
* Graphics: 1 GB of video memory
- Intel: HD Graphics 4000
- AMD: Radeon R9 M290
- NVIDIA: GeForce 640M
* Storage: 5 GB available space

Recommended:

* OS: Sierra 10.12.4
* Processor: Intel Core i7 @ 2.4 GHz
* Memory: 8 GB RAM
* Graphics: 2 GB of video memory
- AMD: Radeon R9 M390
- NVIDIA: GeForce 780
* Storage: 5 GB available space

The list of Apple systems with AMD graphics that support Metal is:

Mac Pro 6,1 (Radeon FirePro D300, D500, D700) - cylinder
iMac 15,1 (Radeon R9 M290, M295X)
iMac 17,1 (Radeon R9 M380, M390, M395, M395X)

MacBook Pro 11,5 (Radeon R9 M370X)
MacBook Pro 13,3 (Radeon Pro 460)

The list of Apple systems with NVIDIA graphics that support Metal is:

MacBook Pro 9,1 (GeForce GT 650M)
MacBook Pro 10,1 (GeForce GT 650M)
MacBook Pro 11,3 (GeForce GT 750M)
iMac 13,1 (GeForce GT 640M, 650M)
iMac 13,2 (GeForce GTX 660M, 675M)
iMac 14,2 (GeForce GT 755M, 775M)
iMac 14,3 (GeForce GT 750M)

If you are looking for 2-3 months of straightforward contract work, are excellent at optimizing art assets in Maya, can pick up workflows quickly and have good attention to detail, then we are looking for you! We are planning on shipping The Witness on iOS in a few months, but still have a lot of optimization work to get done.

Familiarity with The Witness is obviously a plus, as it is a very intricate game with a lot of gameplay-relevant details. Being able to work in-office in our San Francisco studio would also be nice, but is not required if you are really good at communicating long distance.

Please only apply if you have shipped games before and have a good idea of the optimiziation process, we do not have time to bring someone inexperienced into the fold!

Send resumes/portfolios to jobs@thekla.com

To celebrate, the game is on sale for 50% off on Steam, here: http://store.steampowered.com/app/210970/

Braid is also on sale, and if you already own it, you get an extra discount on The Witness: http://store.steampowered.com/bundle/2501/

Meanwhile, we are working on The Witness for MacOS, and it should be done pretty soon. iOS is also coming up but is a bit further in the future.

Happy 2017!

(That is all!)

That was a work-in-progress, and I said at the time I'd post final specs as soon as we know them, which we now do.

We are definitely supporting HDR, on both the PS4 Pro and the base model PS4. If the game detects an HDR TV, it will start in HDR mode. HDR works in all resolutions mentioned below.

If you have a PS4 Pro, and are outputting to a 1080p TV, The Witness will render at 1080p, 60fps, 4x MSAA (which is substantially higher than the base PS4's 900p, 60fps, 2x MSAA). This is the same as we announced in the earlier posting.

If you have a PS4 Pro, and are outputting to a 4k TV, then you get a choice of two modes from the options menu:
(a) 1440p 2x MSAA rendering of the game world, upscaled to 4k, text and UI rendered at 4k, 60fps
(b) 4k 2x MSAA rendering for everything, 30fps 

We added option (b) because a lot of people seem to want it, and it does look really good!

I don't yet know the exact release date for the patch, but it is coming pretty soon, in November sometime. We are now in the process of nailing down all the little details and making sure they are good. As soon as we know the release date I will announce it here.

If you have an Xbox One, you can play The Witness there! Unless you are in Australia or New Zealand ... in which case we need to get the ratings stuff pushed through. We are working on that.

The Witness already runs at a consistent 60fps on the base PlayStation 4, so rather than increasing frame rate, our patch is about visual quality. Please don't take the specifics mentioned here as absolute promises; things might change by release. But here's what we are aiming for:

When you start up the game on a PS4 Pro, it will go into one of two modes, depending on the current display device:

* When using PS4 Pro with a 4k display, we will render at 1440p or possibly higher, all while keeping a consistent 60 frames per second. This image will then be upscaled to 4k, and we will draw text, menus, and other UI at native 4k resolution. (This is similar to what happens on the base PS4, where we render the 3D scene at 900p, then upscale to 1080p and draw UI at 1080p). We might be able to achieve resolutions higher than 1440p based on various engineering tradeoffs (for example, if we boost the resolution of the main scene to a greater degree than we boost the resolution of the reflections in water, which tend to be distorted anyway. Also, we'll evaluate checkerboard rendering -- we're not sure if it's the right way to go for this game, but we'll see.) We'll post again when we finalize these specs.

* When using PS4 Pro with a display that is 1080p or lower, we will render the scene at 1080p (instead of the old 900p), and increase antialiasing quality from 2x to 4x MSAA, again while keeping a solid 60 frames per second. We are exploring additional visual quality increases, like increased streaming distance for high-resolution meshes, or decreasing the noticeability of the fade-in between faraway LODs and high-res close-up meshes. Again, we'll post details when this is finalized!
We're also working on HDR support. Here in the office we just got a new LG B6P television, which is very bright and has low black levels, providing a lot of dynamic range, and it's been fun so far tweaking the look of the game on HDR displays like this.

We thought we were working on HDR support only for the PS4 Pro, but yesterday Sony announced that they will be enabling HDR support on all PS4s via a firmware update. Sometimes Sony even surprises developers! (Back when the PS4 was originally announced, we were surprised by some of the final specs too ... and I was on stage in that show!)

We think it's great that Sony are doing this. We don't know the technical details of how it will work, yet, but provided that nothing prevents us, the Witness patch will use HDR on all PS4s when available. (You will, of course, need an HDR TV or monitor for HDR mode to kick in!)

If all goes well, this patch will be downloadable on November 10 when the PlayStation 4 Pro is released.

We'll keep you posted as we continue work on this patch!

We are looking for a senior character artist, full time or contract, who excels at both designing and modeling intriguing characters in any style, and can also help out with environment art if the need arises.

We would prefer applicants who are very comfortable making 3D games from white box to polished product, in Maya/Max and in-engine. If you apply, you should be someone who can work in interesting, creative styles, and have the imagination and self motivation to take an abstract mockup and turn it into something stunning and meaningful, while also paying close attention to gameplay limitations.

Good communication skills and the ability to take and incorporate critique is a must, a generally pleasant and chill nature is a plus.

We would prefer someone who is based around San Francisco and can work in the office.

Please send portfolios to jobs@thekla.com

Hey there! Lately I've been seeing a lot of requests from people who want to buy framed panels, like the ones we have hanging in our office. We talked a little bit about how we might sell these with Jon, but decided it would be best to just release the images we used to print ours for free, and that way peeps can print their own however they like.

An important suggestion would be to print these on semi-glossy or glossy paper, as the colors tend to turn out completely faded on regular paper. Back when we were printing these, Luis tweaked the colors for some so they look better printed, so they might not be completely accurate on a monitor, but should look nice on paper.

Also, we just went to a nearby art store and picked out a bunch of square frames. We got mostly 8x8 inch and a couple of 12x12 inch ones. Most of the frames were just minimal black plastic, and only cost about $6, if I remember correctly, but we got nicer ones for a couple of the 12x12 ones, like the boat panel. You could probably even make Witness monitor style frames for them if you are crazy enough, that would be fun to see!

Here are the files we used:

Click here to view the video on YouTube.

 (Alcohol Reference)

This will be Friday, February 12th, starting at 1pm Eastern (10am Pacific) and going for ...  a while?

I have made a new reddit account with the fabulous name of "Jonathan_Blow" for these purposes. (Note the underscore! There are some other accounts with similar names!)

Maybe we can also get some other people from the team in there answering questions.

The thread is now up.

There is a wrinkle, which is that I can't be too precise about the sales results on specific platforms, because often when we sign a deal with a particular store, we agree not to reveal their sales numbers. I am actually not sure at this time which stores we are allowed to be specific about and which we aren't, and I would have to dig up and sort through a number of contracts to be sure about it; but that is not a good use of time right now, since I am spending most of my day supporting the users who have technical problems (shipping games on PCs these days is really not fun). The situation can be thorny in subtle ways, too, because if I post information about all stores but one, then I am implicitly revealing the sales figures for that last store, which is not allowed.

Finally, I want to make clear that we did not make this game in order to make money. We were trying to build a beautiful / interesting / intricate thing, first and foremost. The money just helps us stay in business in order to build new things. It is very easy on the Internet to read a financial posting like this cynically, so I urge folks out there not to do that.

Okay, so here's what I can say:

Across all platforms, The Witness has totalled over $5 million USD gross revenue in the first week, and it has sold substantially more than 100,000 units.

This is a good chunk more revenue in one week than Braid made in its entire first year, from August 2008-September 2009. (Braid initially launched on XBLA in August 2008, and it came to Steam in April 2009). Braid was considered a hit independent game at the time.

We can also compare sales by units instead of revenue; this is a little more of an apples-to-oranges comparison because The Witness has a higher price than Braid did (Braid launched at $14.99 [$16.50 when inflation-adjusted to 2015 dollars], and The Witness launched at $39.99). By number of units, the first week of each individual platform handily beats Braid's first week of sales. (Witness on PC by itself beat Braid's first week by a decent margin, and Witness on PSN by itself beat Braid's first week by a decent margin, counting only by number of units). This is great because as price goes up, naturally the number of units sold goes down. So the fact that we beat Braid by units, more than doubly, is a really nice success.

The Witness launched on two platforms, PSN and Windows PC. Neither of these platforms dominates our sales; PC is very strong for us, and PSN is very strong for us.

There are some publicly-available guesstimates for specific platforms on sites such as SteamSpy, but the numbers that SteamSpy is reporting for The Witness are a bit too low (though this is kind of to be expected, I guess, from the way that site works).

So, the game is doing great. That doesn't mean we have broken even on our development cost yet! Because our development budget was so high, $5 million in revenue is not enough to recover it yet (because we split that revenue with the storefronts, we have to subtract VAT in Europe, etc). However, it is looking like, as time goes on, we should break even and make a comfortable safety margin on top of that, which will allow us to make more nice games in the future -- unless some kind of world economic disaster happens.

As I mentioned, right now we are dealing mostly with PC graphics driver problems, and we are also working on adding some features to the game about configurable controls and rendering options, for PC and PS4.

After this, in the near future, we will start investigating the bringing the game to other platforms. Under serious consideration are: iOS, Android, Xbox One, OS X. We will provide more-concrete information about these as it becomes available!

It should go up on Steam this morning around 9 or 10am Pacific time.

I made a posting on the PlayStation Blog: https://t.co/ChsVlBzhVw

Thanks to everyone who's been following along on these forums, and everyone who's been interested in the game generally!

We'll see what happens tomorrow!

A lot of people have been asking about preorders too. Preorders will go live on Steam and the Humble Store (and on this web site) today at noon PST. Unfortunately we won't have preorders on PSN, because apparently developers have to set those up months in advance and we didn't know that!

The preorders are regular price! They contain an amazing bonus pack-in: the warm fuzzy feeling that you have pre-ordered the game. (And that's all). So it's exactly the same as buying the game at launch, except earlier.

If you buy from the Humble Store, 10% of the price goes to the Against Malaria Foundation. For a lot of us in developed nations, malaria seems like a far-away thing that you only think about when you go on a really adventurous trip, but in many parts of the world it's a serious problem. The good news is that we are making great progress in reducing incidence of malaria, see this report:

The Against Malaria Foundation is rated one of the most cost-effective charities in the world by both Giving What We Can and Givewell, two organizations focused on ensuring that charitable donations do maximum good in the world. Giving What We Can has a good high-level description of the philosophy behind this movement.

The Witness deliberately riffs on several traditional elements of game design, and one of those elements is the classic audio log, where you are wandering along and you find a recording that was placed there by someone before. The Witness has recordings like this, but let's just say they are done in a highly non-traditional way. When you first encounter them you won't find much in the way of answers, but, if you keep looking, you will eventually be able to piece together what's going on.

These are the folks whose fine voicework you will hear in these recordings:

Ashley Johnson, who you may know as Ellie from The Last of Us, or from her roles in Tales from the Borderlands and Infamous First Light. She also plays the smartest person on the hit TV show Blindspot. You can find Ashley on Twitter.

Phil LaMarr, who I best know as Marvin from Pulp Fiction, but who you may know as Vamp in Metal Gear Solid 2 and 4, or as Ratbag from Shadow of Mordor. He also played Hermes in Futurama and Ollie Williams in Family Guy! You can follow his random musings here.

Matthew Waterson, who played very English person Sir Lucan in The Order: 1886, an has recently acted in Fallout 4 and Halo 5. Check out Matthew's amazing Sean-Connery-themed profile picture.

Terra Deva, an oft-performing musician, including recent hit single "At Night" with the swiss group Shakedown (though she doesn't play music in The Witness because we don't have music in The Witness! She is acting!) and veteran of... The Mickey Mouse Club!!!! (Among many other places!) Terra's Twitter page is here.

We recorded these sessions at Warner Bros. Studios. Our voice director was Liam O'Brien, who has also done quite a bit of acting in games himself. Pierce O'Toole acted as the great force of organization throughout the process and many things would have gone worse were it not for him!

Having said all this, I will warn you that the story in The Witness, such as it is, is fairly subtle and happens mostly implicitly. You really have to explore and read between the lines to know what's going on. So do not expect a traditional game story like The Last of Us or The Order! All that said, we are very happy to have these actors lending their talents to our game, and I look forward to release when you'll see what we've been up to.

Well, I think we will post a little more frequently, because we'll have increasing amounts to say about the game as we get closer to release. In the meantime, here's:

(It is longer than Long Screenshot #1!)

Voiceover

This week we just finished recording the majority of the final voiceover. We're going to do a little more in early January, but most of it is in the can. After New Year's we'll make some postings where we announce who the voice actors are (you may know some of them from other recent games!)

Physical Box Release

We've had some people asking us whether there will be a physical retail release of The Witness. It's seeming likely, as we have been in talks with a couple of parties for a while about that possibility. *However*, it looks like if there is a retail release, it won't happen until a little bit after the digital release. The reason is just that we need all the time we can get up until the release date in order to get all the finishing touches into the game; but because of the lead-time involved in physical manufacturing and distribution, we would have to freeze the game right about now in order to have it on store shelves for the release date. *Or* we could delay the digital release date in order to wait for the physical. But I don't see a good reason to do either of those things. I would rather just focus our energy on making the game as good as we can make it, and a retail version can lag behind by a little bit.

What Else is Going On

We've gotten the bulk of the translations into the game for all the languages mentioned in the previous blog posting. There is a small percentage of text that will come in a little later, but for the most part those languages are in. Also, we will be adding Hungarian to the list of languages suppoorted at release.

Here are some screenshots of Arabic and Russian:

Of all the languages, supporting Arabic took by far the most programming work, and as you can see from some gaps between the letters, and a couple of cases where letters are too close to each other, it's not quite done yet. I will be working on that as soon as I finish with this blog posting! But I think even with these issues it is starting to look nice.

We're doing *lots* of little gameplay fixes all around the island, like, what does collision feel like in this area when you bump into things, what happens if you are standing in this doorway and the door tries to close on top of you, all that kind of thing.

Visuals are mostly locked down at the moment; we are no longer changing big things about how the game looks, we're more making spot-fixes here and there every time we see something we don't totally like.

Music and Sound Effects

Once in a while I get a question, from someone who really liked the music in Braid, what the soundtrack in The Witness is like. The answer to this question is that there isn't one. There is (almost) no music in the game. This is not an arbitrary decision, but is in fact very important to the coherence of the thing we are making.

The Witness is a game about being perceptive: noticing subtleties in the puzzles you find, noticing details in the world around you. If we slather on a layer of music that is just arbitrarily playing, and not really coming from the world, then we're adding a layer of stuff that works against the game. It'd be like a layer of insulation that you have to hear through in order to be more present in the world.

Instead, we put a great deal of care into the sounds of the world around you, in a way that maximizes immersion in the game. This is made trickier by the game's setting: you are alone on this island, and there are not even any other animals. There are no birds in the trees! In everyday life if we imagine the sound of nature, we'll think of some elements that have no place on the island: a forest naturally has the sounds of birds, plains with smaller shrubbery will have crickets, a marsh will have the sounds of many insects. There's none of that in this game because in this game you are really alone, and it has forced us to be very creative with the audio in order to ensure things have depth and texture to them. This work is being done by Wabi Sabi Sound, who did the sound for the very atmospheric Dead Space series, and more recently some smaller, artier games like Ori and the Blind Forest.

We thought it would be fun to release a few more small videos to give you more of a feel of what the world of the game is like, so here's:

Story and Voicework

Lately I have been doing a lot of work on story and voice acting stuff. When wandering around the island, you may find voice recordings that were placed by ... well, at the outset of the game, you don't know who; and as the game goes along, there is an interesting mystery to unfold about who these people are or were. In the game currently we have what is known as "temp voice", which is me reading all the parts under very noisy conditions and just getting them into the game so we can make sure the data works properly with the engine, to get all the subtitles into the game and roughly synced with the audio (which we need to do sooner than later so we can give the translators those subtitles, which they will translate into many different languages).

We have cast the real actors and starting tomorrow we'll do our first test readings with those actors. We'll use this to get a new layer of temp voice in the game, which will get it closer to final; this will allow me to see how the game really feels with this stuff in there, possibly to adjust the nuances of the way the recordings appear in the game and the way they are played back. When everything feels good we'll do a set of final recording sessions. There's not a whole lot of time until release, and we have several holidays coming up, but we'll get it done!

We are working with Warner Brothers Game Audio in Burbank, California to do the voice stuff; they are providing the facilities and voice direction and production assistance. After we do a few recording sessions I will have more details to share in this department.

Localization

Earlier, I mentioned subtitles. As many games do, we will be localizing our game so that people who speak many different languages will be able to understand the words. We are not recording voice acting in different languages, mainly because it is pretty hard to make sure quality is high when you do that. It is hard enough making sure acting is good in just *one* language. We generally seem to feel the same way about movies: a foreign-language dub of a movie is generally considered to be inferior to a subtitled version, because the acting is so important.

Our localization efforts will involve menu text and subtitles, just as for a good movie.

This is not 100% final, but here's the list of languages we are planning to have translations for, at launch:

• English
• Italian
• French
• German
• Spanish (Spain)
• Spanish (Latin America)
• Portuguese (Portugal)
• Portuguese (Brazil)
• Arabic
• Russian
• Chinese (Simplified)
• Chinese (Traditional)
• Japanese
• Korean
• Polish
• Indonesian

We may add more languages later but man this stuff is expensive, so there is a limit to how many languages we can do at once.

We will have more news for you before too long!

You can read it here.

Here's my update on the PlayStation Blog.

Also, we have some interviews and previews of the game:

Gamespot
Ars Technica
Polygon
Wired
PlayStation Blog

The PlayStation Blogcast had a nice ten-minute discussion of the game.

It will not only show the game in a further state of visual development than the previous trailer (which is two and a half years old now -- wow!), it'll also show lots of scenes you probably haven't seen before.

The trailer isn't done yet, but here are a few stills:

Ignore the debug text in the upper-right (including the frame rate; these were taken on a monitor that's locked at 30Hz; the game runs much faster on higher-end PCs.) Also, the actual trailer will be produced at much sharper image quality and higher resolution. These are just snippets of the non-final footage we are using to figure out the composition and flow of the trailer.

A few months ago, only a few of Jon's friends were playing the game and checking for major problems, but now we have Francis as a game tester, and we have an other guy starting later this month as a tester too. With someone testing the game full time, the bugs are really piling up. This is great because we are getting the game in shape for release, but it's also wearing us down a bit because it's non-creative, discouraging work, since for every bug we solve there are usually more that appear on our list.

There are many different kinds of bugs, some really major and a ton of minor things that can be solved fairly quickly. We are finding errors of all sorts, in lightmaps, shadows, collision, grass planting, reflections, mesh placement, modeling, materials, sound, etc. Not to mention a bunch of weird things that sometimes take a while to track down. We also have to check the game in High, Medium and Low quality, because different errors can happen at the different settings.

The biggest bugs involve spoilers, since they are usually finicky and gameplay related, so I can't share those, but here are some minor ones that wouldn't generally take too long to fix:

-Here we have a bit of collision randomly hanging out in the desert. If you tried to walk through it, you would be blocked and have no idea why. Things like this were really difficult to find, until Casey added some tools to visualize the collision:

-Sometimes, we just get bugs of areas that look unfinished, or wrong, like this corner over here. Francis is supposed to tell us not only about serious errors, but also notify us about things that generally feel wrong to him as a player. By fixing these areas, we hopefully get rid of some potential immersion breaking moments.

-In this shot, some of the little modular rocks were not receiving any sunlight:

-And here we have two separate bugs, the turret that is getting black lightmaps, and an open rock mesh below:

-This is a shot of the player being able to walk into a rock, and seeing through it:

-This is a pretty one, some error with the clusters of the modular catwalk meshes:

-There are tons of cases where the grass is clipping through things:

-And little lightmap errors that mean there is a back-facing triangle somewhere in the mesh:

-In this area the dynamic lights create some really ugly shadows:

-And finally, the bug that haunted us for at least a year, the giant colorful grass bug. It would happen almost always when replanting grass, and would not disappear until you restarted the game. Insanely large, colorful versions of the grass would appear and wave in the wind, blocking you from doing anything. Still not sure if we ever figured this one out:

These are only a tiny sliver of all the different kinds of things that we run into while testing the game. It's a huge island and a lot of things can go wrong, so to make sure we ship a great game, we have to do a really thorough job. However, this and some optimization is pretty much all that's left to do!

And now, here is a bonus a picture of Jon doing something weird with a tape measure taped to a mug:

Here is an example, a set of stairs added to a roof structure:

It looks ok, but looking close up you can see how flimsy the roof attachment is:
Or the way the railings are assembled together:

Deanna showed us that it would make more sense to extend the main roof support to accommodate for the stairs:

An in the case of the railings, to do a better job showing the assembled pieces, removing the awkward intersections:

Another example is this stone wall door frame. It lacks any assembly detail for its size, making it feel it was made from one single gigantic piece of stone:
And here is the revised version:

Another type of issue is how two different surfaces meet. The more detailed the island gets, the more these stand out as incomplete. As an example, the way this wood structure connects with the floor:

And after the revision:

Some of this work might seem minute but it really makes the whole environment feel more grounded, adding a lot to the experience!

We managed to find Andrea Blasich, a most excellent artist skilled at both traditional and 3D sculpture. He is blowing through the statues and continues to wow us with his stuff. He just finished this giant lady sculpted into the mountainside, and it turned out so beautiful, I wanted to share it:

When we moved in two years ago, we just pulled things out of the moving boxes and shoved them here and there so we could get to work as quickly as possible, so the mess was only getting worse. We spend most of our waking lives here, so I thought it was worth a few hours to make it a more pleasant space. I moved some furniture around to create a more convenient place to test the ps4 build, threw out a giant pile of old things, put up some of the decorations we made for the Play Station Experience booth, etc.

I figured I'd put up some pictures, so you guys can see where The Witness is getting finished (because it's getting finished, yo!):

You can use Oswin, a.k.a. The Witness Dog, for spatial reference.

It does *not* mean that the game is done. We still have a lot to do! But it *does* mean that the nature of the work changes and becomes simpler, because we don't have to be making high-level creative decisions any more. It is now much more about turning the finish-the-game crank (making sure stuff plays well and polishing it up) for anything related to game design, modeling and texturing.

There are a few categories that still need creative decisions (story stuff, the menu system, etc), but for those of you who are awaiting the game's release, this is pretty good news.

There are currently 677 puzzles in the game. That number might be slightly different when it ships (we might cut a few, for example!)

We spent over a week play testing,  bug fixing, designing the booth, printing banners, and re-designing the website. It was encouraging to see that the game was already in very good shape and nothing major was broken...getting it ready for the event almost took less effort than getting decent prints of the banners from a print shop.

Here are some images of the booth:

People loved the game! Even those who came in with doubts ended up playing for long periods of time, and we got all very positive comments. The line in front of the booth was huge, it would have been great to have a bigger area with more PS4s.

We made some new posters while preparing the booth:

• Play
• Fullscreen

If you wanted to print your own posters or bookmarks, you can download the huge versions here.

Jon also did a short Twitch stream, it's on YouTube here.

We are all working hard, hence the silence on the blog. Always appreciate the kind words in the replies, so I wanted to at least add a quick new wallpaper to say thanks. Hope you like it!

We've been revising the modeling and texturing for areas near the end of the game. There are some nice treats in store!

We've been working on some of the "special effects" in the game, most notably particle effects. Our particle system was old and slow, so we are speeding this up, and then we will make some modifications to help make particle authoring an easier process.

We've been making some baseline functionality smoother. For example, so that you don't need to think about saving your game, The Witness autosaves your progress every minute or so. On a high-end PC, you don't notice the autosave, but on a low-end PC or on a console (because consoles tend to have different memory access properties than PCs), scanning the world for stuff that needs to be saved can be pretty slow, like, it could take 1/10 of a second, which would result in a hitch in the frame rate happening every minute. So we are hammering on this to make it faster, which should also result in saved game files being smaller, since we'll be excluding a lot of things that don't need to be saved.

The D3D shader compiler provides a backward compatibility flag (D3DCOMPILE_ENABLE_BACKWARDS_COMPATIBILITY) that I was hoping would allow us to use our shaders 'as is'. Unfortunately, it seems the backward compatibility mode has not received much love and things were not that simple. I run into several compiler internal errors, and even after finding workarounds most of our shaders did not actually compile, but would have required significant modifications, the resulting vertex and pixel shaders would often not link at runtime, and the use of register annotations on samplers only allowed us to control the layout of the samplers, but not of the associated textures.

For other targets (Cg & PSSL) we had been able to get away with some preprocessor macros and simple text transformations, but it seemed that was not going to be an option in this case. Not only that, having to write and adapt our shader code for the idiosyncrasies of each compiler was starting to become a challenge.

Instead, I thought it would be much cleaner and much more powerful to standardize on a fixed subset of HLSL and transform it to the shading language of the target by building a syntax tree, applying syntactic transformations, and generating shader code by traversing the resulting syntax tree.

This is essentially what we already do in the OpenGL backend to transform GLSL into HLSL. For that we use HLSL2GLSL and GLSLOptimizer. I considered using these same projects, either modifying HLSL2GLSL to output HLSL1x code instead of GLSL, or modifying GLSLOptimizer to accept and produce HLSL code. I avoided the former, because HLSL2GLSL's code was a big mess and it was not very attractive to work with it. The latter was more interesting, and I believe it's the approach taken by the UE4 engine. However, while much cleaner, GLSLOptimizer is still a fairly large project and I did not care too much about the optimization aspects that it offered.

Another option that I had just come across is Max McGuire's HLSLParser. HLSLParser was a lot smaller and cleaner than either of those code bases. It was pretty much the code I would have written if I were to do this from scratch, so it seemed like the best possible starting point.

One of the main annoyances when working in the OpenGL backend is that GLSL errors are extremely hard to track down. They can be reported by either HLSL2GLSL, GLSLOptimizer, or the driver's GLSL compiler. Each of the inputs to these compilers has different file and line information and none of them matches the original source code. Whenever an error occurs we have to dump the intermediate code and figure out from there what's going on. It's usually a waste of time and if D3D11 is to become our primary backend I did not want to have to deal with this issue.

I was happy to see that HLSLParser handled this correctly associating file and line info to the tree nodes and producing correct #file preprocessor statements in the generated output.

In order to parse our shaders with HLSLParser we had to extend it in several ways, here's a partial list of the features I added:

• Support for 3D, 2DMS and shadow samplers, and most of the associated intrinsics.
• Added support for many missing intrinsics.
• out argument modifier.
• Default argument values.
• Block statements.
• Attribute annotations.
• Static, inline, uniform and other type modifiers.
• Bitwise operators.
• Hex literals.
• Full declaration syntax, including multiple declarations in the statement.
• Add support for full declarations in buffer fields.
• Effect syntax (techniques, passes, sampler states, state assignments).

I'm very thankful for the work Max McGuire did on HLSLParser and the least I can do is to release these our improvements to the public as well. You can find our fork of the project in our github repository:

https://github.com/Thekla/hlslparser

Once we have the syntax tree in memory we can do many cool code transformations. In particular, we do:

• Dead code elimination.
• Determine used parameters and resources.
• Assign registers/offsets explicitly to control layout of parameters and resources.
• Sort resource registers and group parameters in buffers based on frequency of change.
• Rename semantics.
• Reorder pixel shader arguments so that vertex shader outputs match pixel shader inputs and avoid linkage errors.

The shaders that result from these transformations are minimal, that is, they only contain the definitions and declarations that are necessary for a specific entry point. We have complete control of the layout of resources and parameters and extract all the necessary information about the shader from the tree instead of relying on the platform-specific reflection APIs. The shader transformation is very fast and the resulting shaders compile faster than the original ones. Our new D3D11 shader processor and compiler is about 4 times faster than its D3D9 counterpart.

While it currently accepts all of our shaders, HLSLParser still has some limitations. Here are a few that come to mind:

• No type inference for sampler types. It assumes sampler == sampler2D.
• No support for {} initializers. Instead of float2 p = {0, 0} you have to write float2 p = float2(0, 0). This particular case is not complicated, but it does get more hairy when initializing structures. For now, I've simply modified our shaders to avoid this.
• No support for rectangular matrices. We did not support them in the GLSL translators either, so we avoided their use in our shaders. It would be nice to add this eventually.
• No support for do, while or switch statements. These should be easy to add, but we just didn't need them!
• No proper error recovery. We stop reporting errors after first one. This would be nice, but is not critical.
• Semantic analysis is incomplete. The parser does some, but not a complete semantic check. The generated shader is not guaranteed to be correct in the absence of errors, but since we have proper file/line annotations, this usually doesn't really matter.

Looking forward, I expect we will add code generators to target other platforms. It should be really easy to bring the existing GLSLGenerator up to date, or add new generators for PSSL and Metal.

I'm not particularly fond of our shader code. On one side, we abuse the preprocessor to generate shader variations and the result is often a mess that is hard to read and modify. On the other, our code is often highly repetitive, but there doesn't seem to be a clean way to share it across different materials or structure it in a way that's more compact. I'm particularly interested in experimenting with higher level code transformations, or coming up with new constructs to facilitate writing and sharing shader code.

Ignacio has posted the conclusion of his series on irradiance caching -- at long last! -- and released the source code.

This code is a significant chunk of what makes lighting in The Witness as nice as it is.

Here is a screenshot from another angle:

Most of the key detail has been added to the walls and I feel, if we had to send this build to E3, it wouldn't be terrible.
We managed to get in some really nice framing details that I won't spoil here but I'm sure players will appreciate them!
Here is a shot showing the ruined walls work.

Tomorrow I'll make sure I walk around a lot to hunt down any major issues, get the overall color vibe (mostly get more variation on the walls) and any extra details.

Orsi's Vegetation Update:

To give a little variation to the post today, I thought I would include some things I have been working on as well. While Luis had been working on the structures around the entry yard, I have been placing the vegetation and cables, and re-working some puzzle related things. Laying out the vegetation early on was important to give the place a sense of scale and to get the overall vibe of the place. I am very happy we got an other chance to re-do all of it, because I think it turned out much more magical this time around:

Even though it's a much smaller area than it used to be before, I used a different combination of plants in different areas, so people would be less likely to get lost:

I still have polish to do, to match everything to the walls Luis finalizes tomorrow, and then we should be ready for E3!

The center building is now complete and we got the layout for the ruined walls so the major task for tomorrow is to get them all detailed (very time consuming!)

And finally some new interior spaces that you can't really see from the previous angles.

Orsi is in charge of the vegetation (she is amazing at it) so that is one big chunk of work I don't have to worry about!
I'll be aiming to have everything done by tomorrow so I can use Friday for polishing and any major issues that I might have missed.

Here are the new modular pieces that were done for detailing:

We just finished the day by meeting with the architects and discussing the interior layout of the space so tomorrow I'll start on the interior building and ruin walls.
Here is how it currently looks:

Last week, as you could see from previous screenshot, we got  a rough layout of the area in the engine and tried to find answers to any remaining issues.

Today the goal was to sort out the design  and navigation of the main entrance and the small towers.
This is one of my first mockups that didn't look quite right.

The architects took a look at it and provided some 3d models of how castle stairs are usually assembled as well as sketches of how it should look:

After, I tried to figure out the details from references and what "pieces" I would need to build the structure.
Two years ago I went for vacations in Portugal and tried to see some castles. One of them in Guimaraes struck me as very similar to the entry yard. I made sure I got some pictures and now they are proving to be useful with angles that would be hard to get from online research.

Here is the current result using different modular stairs/floor/battlement pieces:

And the current overall look by the end of today:

Tomorrow I'll be tackling  the rest of the exterior walls and the 3 other towers.

This is of course a very crude mockup of what the new starting area will be. It's smaller than the old one and should work a little better play-wise. We have yet to reshape the terrain around it, so there are huge holes. Our deadline to have this area fully rebuilt is the 6th of June!

Current puzzle count is 627.

http://mollyrocket.com/casey/stream_0019.html

Meanwhile, we've had a lot of Witness Wednesday posts in a row on this blog. I'll have to stop slacking and write something about the game.

Here's the blog post.

Here's the video:

Update: And here's another post, complete with a video showing in detail how the grass planting works in the actual game.

...can't say, but it sure was fun to build!

Working on The Witness, Part 5

Working on The Witness, Part 6

In this one he talks about how to distribute grass. Legally. As well as the different colors of noise one might want to be familiar with in order to do this well.

If you are interested you can take a look at the slides PDF here:
GDC2014_LuisAntonio_TheArtOfTheWitness

Unfortunately the PDF merges animated slides so there is some content that you won't be able to see and you are missing the actual talk but it should be available in a few weeks at the GDC Vault!

*UPDATE - The talk is now available at the GDC Vault, you can  check it here:
http://www.gdcvault.com/play/1020552/The-Art-of-The

Around Christmas time, Jon surprised us with a bunch of new slab buildings full of puzzles in the center of the island.

Previously, there were only some empty ruins here, so the whole area was redesigned by the architects to accommodate the new puzzles. I attacked it first, about 3 weeks ago, adding a couple of puzzles myself and starting to model the three buildings on the right. Now, the entire art team is working on it. Luis is taking care of the entire left portion, which is still mostly low ruins, Eric is tackling the tower in the middle, and Alex is working on the building behind the tower, with the orange box sticking out of it. There is still one slab building up for grabs, but it's coming together very quickly for such a huge area!

Last year I was pessimistic about VR systems in the near future, since the ones I had tried didn't seem to offer much (certainly not enough to justify wearing a bulky headset all the time and fighting back feelings of nausea).

But I was fortunate enough to get a sneak peek at Valve's virtual reality system. It is so much better than anything else I had used that I was instantly very excited by it. Right away I could see games you might design for this system that had been impossible before; so it isn't just a matter of the system being very immersive (which it is) but that it can take video games in general to a new place.

We worked with Valve to get support for the system in The Witness, and that's where those screenshots come from (but when I took the shots, an HMD was not plugged in, and as a result what you see is a dummy calibration that does not correspond to any useful image-warping; the screenshot was really more about just being happy about the stereo display capability.) In late November I spent a few days at Valve's offices working with Atman Binstock and Doug Church to adapt the HMD's input system more thoroughly for The Witness and to build a few scenes that are playable on the device.

(In theory the whole game is playable on the device, but when you are rendering for VR the performance demands become much higher, because you are rendering two scenes at higher resolution than you might previously have been rendering one scene. So the full game world with all content currently does not run fast enough for this device, but that is the kind of problem that gets solved over time).

We had to keep this under wraps, though, because Valve's system was not announced. But this month at Steam Dev Days they revealed details of the system.

For more details on the Valve system, Lee Vermeulen wrote up a nice post of his impressions:

http://www.3delement.com/?p=332

So, we will be supporting this device (and any similar devices) with The Witness. But I am really looking forward to see what gets designed specifically for devices like this.

Luis has been doing a lot of work lately on the mountain and especially the stuff at the top.

Other folks have been doing a bunch of work on things that are not as obvious in this picture!

Current puzzle count: 595

The interviews happened somewhat randomly. A whole floor of the Standard Hotel was dedicated to indie games that would appear on the PS4, and the press wandered around, peeking into rooms and sitting down to play the games when they were available. I snapped a picture of the room where The Witness was shown:

We spent two days showing the game to many journalists. They usually played from the beginning for a while, then Jon took them on a guided tour of an area where the puzzles were starting to get a little crazy, explaining the thought process behind them. It was so great to finally see the game being played! Everyone seemed to love it, too!

During the nights, we were invited to a few events. Sony took the indie developers out for dinner, where I met Shuhei Yoshida, and chatted with the guys from Secret Ponchos about goldsmithing and tattoos. It was a good opportunity to get to know the other developers, and a nice gesture by Sony. The next night there was also a bigger, louder party in the penthouse of a tall building, to which local developers were invited as well. It was so full we could barely move, so I just stood around trying to piece together conversation through the incessant club music.

By Wednesday we were done with the interviews and I had some time to explore New York and feel totally lazy while Jon got some programming and design done in a cafe. I ended up spending most of the day in the MoMA, where there just happened to be a special Magritte exhibition going. Love Magritte! Oh, and I also got to see Sleep No More, which was a really amazing experience!

The most unbelievable part of the trip for me, however, was seeing our artwork displayed on the side of the 17 story hotel. It was difficult to decipher right up close, since it was displayed on huge sheets of LED lights, but it was wonderful to be right there to see it happen! We took a few pictures but they turned out very blurry, so here are some taken by Kevin Geisler:

Only Alex's windmill and my autumn forest and orchard made it onto the building, but I thought I would include all the original images we submitted, since we put some effort into getting them together. These are pretty high res, in case you guys want to print your own posters or crop them down and make some wallpapers:

Click here to view the video on YouTube.

(I didn't name this speech; the title given here was picked by Creative Mornings after the speech was given.)

Click here to view the video on YouTube.

Discussion on PlayStation Blogcast (Witness talk starts around 26:25):

http://blog.us.playstation.com/2013/10/18/playstation-blogcast-094-witness-the-daylight/

(From the way the PS guys talk, it sounds like it is all me making this game, but as always on games of this size, there is a whole team making it happen!)

Interview with Gametrailers, also from the PlayStation event (you may have to sit through a commercial with army dudes to see it):

http://www.gametrailers.com/videos/aeeuwu/the-witness-the-puzzle-of-making-puzzles

The Witness was among the games being shown! We were the only third-party game there. Also shown were a whole lot of Killzone: Shadow Fall, Knack, PS4 versions of Sound Shapes and Flower, and Resogun.

I did some interviews and talked to a number of press people who are very interested in The Witness. I also took some photos.

Brad from Giant Bomb playing The Witness:

The room we were in:

Flower at 1080p and 60Hz, with the new, more-sensitive controller, feels nice to play:

Sound Shapes:

I'm not going to try to address that here, but just as I was writting this Matt Pettineo released a good overview of some of the different techniques available that is a good starting point. Instead, I'll describe some of the problems that I faced when implementing and optimizing our shadow mapping implementation, some of the techniques that I tried and the solutions that worked best for us in the end.

While there are many techniques that I haven't tried and I'm sure our solution could be improved further, for this game I'm pretty comfortable with our implementation in terms of quality and performance.

Jon wrote our initial shadow mapping system a few years ago and with the exception of some small improvements I made early on, it had remained pretty much unchanged until very recently, when it became obvious that shadow map rendering and sampling was too slow and needed to be improved drastically.

The posts where Jon describes our old shadow mapping system are here and here, but to summarize, we were using stable cascade shadow maps, with 4 partitions, distance based selection, and some innovative tricks to reduce unused shadow map space. For filtering we used randomized PCF sampling and a combination of gradient, slope and constant biases to prevent acne artifacts.

Filtering

One of the first thing I changed in the original implementation was the filtering method. Initially we used a Poisson distribution of PCF samples that was rotated randomly, similar to what is used in the Crysis engines. This approach has many advantages: it scales well reducing the number of samples, the kernel radius can be adjusted easily, but it often resulted in noise artifacts.

In many games, these artifacts would not be very noticeable in the presence of detailed textures and normal maps, but in The Witness the artifacts really stood off in contrast with our stylized geometry and smooth precomputed lighting.

We experimented with higher number of samples, but in our case the number of samples required to obtain smooth results defeated the purpose of using a sparse distribution. We were able to obtain much higher quality results with a compact box filter such as the one described by Bunnell and Pellacini in Shadow Map Antialiasing. The box filter produced smooth results, but didn't do a good job smoothing out the stair stepping artifacts resulting from the limited resolution of the shadow maps. A straightforward extension of their method that looked considerably better is to weight each of the samples using a Gaussian curve.

The naive implementation of this filter uses (N-1)x(N-1) PCF texture samples. In "Fast Conventional Shadow Filtering", Holger Grün shows how to reduce the box filter to (N/2)x(N/2) samples, but argues that separable filters require (N-1)x(N/2), and proposes to use Gather4 in order to reduce the number of samples down to (N/2)x(N/2). I think Grün was trying too hard to sell the Gather4 feature and didn't fully explore the bilinear PCF sampling approach. His approach to bilinear PCF sampling is too complicated and does not result in the lowest possible number of samples. Here I'll show how to compute the filter weights in a more efficient and much more simple way.

Let's first imagine a symmetric horizontal kernel q with 5 taps:

  q = {0, a, b, c, b, a, 0}

Each of these kernel weights corresponds to a bilinear PCF sample, that is, the actual footprint of the kernel is 6 shadow map texels and each of the kernel weights is a linear combination of our original kernel:

  k[x_] := s*q[[x]] + (1 - s)*q[[x + 1]]

where x is the integer texel coordinate and s is the fractional part.

The traditional way to apply this kernel is by simply taking 5 bilinear samples and weighting each one by the weights in q. However, you could achieve the same result taking 6 point samples and weighting each of them by the factors in k.

The important observation is that we can reduce the number of samples to a half (just 3 in this case) by spacing out the PCF samples so that their footprints do not overlap and adjusting the subtexel location of the samples and the corresponding weighting factors. That is, for every sample we want to find a new subpixel coordinate 'u' and a new weight 'w' that meets these constrains:

  eq[1, x_] := (k[x] == (1 - u) w)
  eq[2, x_] := (k[x + 1] == u*w)

Solving this equation system in Mathematica gives us the following result for each of the 3 samples:

  Simplify[Solve[{eq[1, 1], eq[2, 1]}, {u, w}]]

    u -> (b + a s - b s)/(a + b - b s)
    w -> a + b - b s

  Simplify[Solve[{eq[1, 3], eq[2, 3]}, {u, w}]]

    u -> (b - b s + c s)/(b + c)
    w -> b + c

  Simplify[Solve[{eq[1, 5], eq[2, 5]}, {u, w}]]

    u -> (a s)/(a + b s)
    w -> a + b s

This may be equivalent to what Grün does to half the number of samples in each of the rows in the kernel, but contrary to his method, this can be extended to separable 2D kernels.

Let's say we have a separable 5x5 filter like this:

  q = {{0, 0, 0, 0, 0, 0, 0},
    {0, a*a, a*b, a*c, a*b, a*a, 0},
    {0, b*a, b*b, b*c, b*b, b*a, 0},
    {0, c*a, c*b, c*c, c*b, c*a, 0},
    {0, b*a, b*b, b*c, b*b, b*a, 0},
    {0, a*a, a*b, a*c, a*b, a*a, 0},
    {0, 0, 0, 0, 0, 0, 0}}

In this case, the filter touches a 6x6 range of texels and each of the weights is a bilinear combination of the 4 nearest factors:

  k[x_, y_] := s*t*q[[x, y]] +
    (1 - s)*t*q[[x + 1, y]] +
    s*(1 - t)*q[[x, y + 1]] +
    (1 - s)*(1 - t)*q[[x + 1, y + 1]]

To find the new subpixel coordinates 'u' and 'v' and the corresponding weight 'w' we have the following equations:

  eq[1, x_, y_] := (k[x, y] == (1 - u) (1 - v) w)
  eq[2, x_, y_] := (k[x + 1, y] == u (1 - v)*w)
  eq[3, x_, y_] := (k[x, y + 1] == (1 - u) v*w)
  eq[4, x_, y_] := (k[x + 1, y + 1] == u*v*w)

Solving this equation system for the left-top sample gives us the following result:

Solve[{eq[1, 1, 1], eq[2, 1, 1], eq[3, 1, 1], eq[4, 1, 1]}, {u, v, w}]

    u -> (b + a s - b s)/(a + b - b s) 
    v -> (b + a t - b t)/(a + b - b t) 
    w -> (a + b - b s) (a + b - b t)

We can repeat this for the remaining 8 PCF samples to compute the corresponding uv coordinates and weights. Looking at the results we can notice some common terms that appear due to the symmetries in the kernel. Once we factor the common sub-expressions and replace the a,b,c constants with scalars we can simplify the formulas and end up with fairly simple expressions. For example, in our case, in the 5x5 kernel we set the constants to 1, 3 and 4 to achieve a crude approximation to a Gaussian filter and this is the resulting code:

    float uw0 = (4 - 3 * s);
    float uw1 = 7;
    float uw2 = (1 + 3 * s);

    float u0 = (3 - 2 * s) / uw0 - 2;
    float u1 = (3 + s) / uw1;
    float u2 = s / uw2 + 2;

    float vw0 = (4 - 3 * t);
    float vw1 = 7;
    float vw2 = (1 + 3 * t);

    float v0 = (3 - 2 * t) / vw0 - 2;
    float v1 = (3 + t) / vw1;
    float v2 = t / vw2 + 2;

    float sum = 0;

    sum += uw0 * vw0 * do_pcf_sample(base_uv, u0, v0, z, dz_duv);
    sum += uw1 * vw0 * do_pcf_sample(base_uv, u1, v0, z, dz_duv);
    sum += uw2 * vw0 * do_pcf_sample(base_uv, u2, v0, z, dz_duv);

    sum += uw0 * vw1 * do_pcf_sample(base_uv, u0, v1, z, dz_duv);
    sum += uw1 * vw1 * do_pcf_sample(base_uv, u1, v1, z, dz_duv);
    sum += uw2 * vw1 * do_pcf_sample(base_uv, u2, v1, z, dz_duv);

    sum += uw0 * vw2 * do_pcf_sample(base_uv, u0, v2, z, dz_duv);
    sum += uw1 * vw2 * do_pcf_sample(base_uv, u1, v2, z, dz_duv);
    sum += uw2 * vw2 * do_pcf_sample(base_uv, u2, v2, z, dz_duv);

    sum *= 1.0f / 144;

The main advantage of the PCF separable sampling approach over Gather4 is not just that it works in a wider range of hardware, but that is also more efficient. With PCF sampling we need to compute the new sampling coordinates and weights, while with Gather4 we have to perform depth comparisons and filtering in the shader. The latter uses a lower number of instructions, but consumes more bandwidth between the ALUs and texture units and more importantly, it requires more registers to hold the results of the texture samples. That said, despite the lower performance, the use of Gather4 has some advantages that we will discuss in the next section.

For those that simply want to use our shadow sampling code without deriving formulas for their own weights, you can find our implementation integrated in Matt's shadow mapping example under the "OptimizedPCF" option. Running his demo in the two GPUs that I have available, a NVIDIA GTX 480 and an Intel HD 4000, I've measured that our technique is between 10 and 20% faster than the Gather4 method.

Acne

Shadow acne is one of the major issues of most shadow mapping implementations. Eliminating shadow acne is such a pain, that I wonder why it's not received much more attention.

In our case I ended up employing what I believe is the most common solution: A combination of constant, slope-scale, normal and receiver plane depth biases. Finding the right magnitudes for each of these offsets is complicated. If the offsets are too big, that results in peter panning or light leaking artifacts, if the offsets are too small, they don't eliminate acne entirely.

In Eliminating Surface Acne with Gradient Shadow Mapping Christian Schuler proposes the use of linearly filtered depth values and fuzzy depth comparisons to mitigate the acne artifacts. However, these techniques are not possible when using PCF sampling.

Another alternative is to render backfaces to the shadow maps, but this approach is more prone to light leaking and does not really solve the acne problems on thin walls. Dual or midpoint shadow maps are interesting, but it seems like they would be more expensive, do not entirely solve the problem near silhouettes or with very thin objects, and impose additional modeling constrains.

The magnitude of the offsets is proportional to the size of the texels and the filtering radius, so a simple way of reducing them is using higher resolution shadows, but larger shadow maps also reduce performance. In the end we just try to find the best compromise between the two forms of artifacts and as a last resort constrain the modelling to prevent the most egregious artifacts.

One of the unfortunate disadvantages of our shadow filtering method is that we rely on the hardware texture units to sample 4 shadow texels simultaneously and perform the depth comparison. When using receiver plane depth bias we would like to use a different bias for each of the texels, but there's no way we can provide a depth gradient to the texture unit. As a result, the receiver plane depth bias is not very effective with our method. In fact, it's so ineffective, that we chose to disable it entirely and compensate with a larger normal and slope-scale bias.

For these biases to work well it's important to scale their magnitude correctly based on the surface normal and light direction. In particular, the slope-scale bias should be proportional to the tangent of the angle between the normal (N) and the light direction (L), and the normal bias should be proportional to the sine:

float2 get_shadow_offsets(float3 N, float3 L) {
    float cos_alpha = saturate(dot(N, L));
    float offset_scale_N = sqrt(1 - cos_alpha*cos_alpha); // sin(acos(L·N))
    float offset_scale_L = offset_scale_N / cos_alpha;    // tan(acos(L·N))
    return float2(offset_scale_N, min(2, offset_scale_L));
}

Finally, one of the disadvantages of the normal bias is that it does not only bias the depth, it also offsets the shadow. On its own this offset is not very noticeable, but the offset amount is different for each of the cascades and that results in an obvious shift between cascades. Blending the cascades helps, but to reduce the problem further we also interpolate the magnitude of the bias from one cascade to the next, so that the shift is barely noticeable.

Coming Next

In part 2, I'll describe our cascade placement algorithm and finish with the optimizations that we use in order to speed up shadow map rendering.

The process for building this ship was interesting. We have architects helping us with building designs, but even the architects were not entirely familiar with the structures of ships. We spent a lot of time looking at reference images, talking about the materials and how the entire structure would decay over time. They gave me a base design for the hull, deck, bridge, and the inner beams of the ship. I modeled that whole, then split it up into five pieces.

The front, middle, and back pieces were kept somewhat intact, the rest was split into scattered rubble to dress the scene. Next, it was time to figure out the gameplay and decide what parts of the ship would be accessible and what parts would be rotten away. Because the ship was so huge and I was modeling it alone, I had to be careful about limiting what would be navigable and what would be seen, since I wanted to keep my workload to a minimum while still having very interesting, believable areas. I spent a long time cutting holes, bending floors and ripping out beams.

Then, I had to figure out how to make a decaying heap of metal with our simple, clean style, as described by Eric in the last post. My first attempts had minimal detail in the geometry, with a lot of noise and contrast in the textures. It was exactly the opposite of what we usually do, but I was trying to develop a process for detailing the entire ship quickly, and it's a lot faster to use blendmaps to add noise than to tediously cut and shape geometry. It looked terrible, but I was at a loss for how to make it better quickly. Fortunately, Luis and Eric encouraged me to just reduce the contrast in the textures, and let the small details in the modeling stand on their own. It took ten seconds to lighten some textures, and all of a sudden I had something I was finally happy with. At first it felt wrong to leave large surfaces so plain, but the other artists were right. With our beautiful lightmaps and simple style, a little detail and a few modular rubble and rust pieces were all that was needed to make the space feel like it belonged to our game.

This shipwreck is still not quite finished, but it is the last huge project on my plate before moving on to the many smaller projects and tasks we still need to complete to ship the game.

In the olden days of The Witness art development, our visual “style” for the game hadn't really coalesced yet.  Or, to be more accurate, we knew what our objectives were (from a purely theoretical, gameplay-based perspective) but we hadn't really grasped what the ramifications would be in terms of our production process.

For instance, on an early attempt at adding a rough-stone wall, I approached it like most modern game artists would: using the full spectrum of art tools and tricks...  normal mapping, edge-distress, enhanced details, etc..

But the problem with this approach is that it’s sort of at odds with one of our prime directives from Jonathan:  to build a game world without unnecessary visual clutter.  “Noise” was our enemy.  But that’s kind of an interesting riddle, since most game engines and art techniques are so often all about adding extra levels of noise, grime, texture, detail, (realism!), etc.

So we began an exploration of different techniques we could use to achieve a sort of stylized realism.  We needed to be able to capture the “essence” of our reference imagery while not being quite so literal.  It also had to be free of gratuitous visual noise, plus appealing to look at and explore.  So what follows is an early effort to break down and re-think that art process in a different sort of way.

In this case, my objective was to figure out how to attack a medium-sized cliff face while avoiding an overwhelming amount of noise/detail.  First off, reference.  Starting with some photo reference, I wanted to isolate the primary features, and not worry at all about the small, noisy stuff:

Then, an attempt to stylistically sculpt those simplified shapes:

 And finally, breaking those forms down into modularized, reusable meshes that could be combined to form a cliff:

One of the biggest lessons we learned from this experiment was that if we give the engine nice clean, simplified geometry (not worry as much about the texturing - just keep it understated and focus more on the overall color and balance), the lighting results we get are gorgeous.  In a way this is extremely freeing, as it lets us concentrate on the overall form of objects, rather than obsess over details like perfect seamless, UV mapping, limited texture resolution, normal mapping issues, etc.  We also learned very quickly that hard edges are not evil....  In the absence of heavy texture detail, the faceting had become a powerful tool to help define the form in a clean, almost graphic style, with minimal noise, but lots of subtlety.

That process of “analyze, deconstruct, simplify” became our new way of approaching complex surfaces.  And so, as we built out more areas of the island, this visual language began to take shape,and you can now see it deployed across most of the rock forms in the game...

Take note that foliage has a different language [see Orsi’s previous post on trees], as does rolling terrain, and hard architectural surfaces.  But this style contrast is extremely useful from a game design perspective, as it helps inform the player as to what materials the surfaces they are looking at are likely made from, without having to resort to overtly photoreal textures.

Even older areas of the game (that were suddenly “out of style”) were able to be refreshed using this new lens, and brought in-line alongside the cleaner, less-noisy artwork:

And here’s some additional rock shots from around the island, just because...

Luis’ awesome sculpted sandstone cliffs:

Orsi’s awesome lakeside steppes:

And a bit of mysterious tunnel:

That’s all I've got to share for now.  Have a great September!!!

-eaa

Normally I would post an Island Snapshot, but the viewpoint we have been using for Island Snapshots is a little bit unfortunate lately, since the main thing taking up the view is the mountain, which is an area we haven't worked on modeling-wise lately. (It is a big area so we are going to do a concentrated push on it in a little bit, when some other things get locked down.)

So, here are some screenshots of other stuff we have been working on lately. Both areas are in progress and will look better than this in the final game!

A lake that's central to the island...

A swampy area with light buildings and walkways. The vegetation you see here is temporary! But I am happy we are finally starting to work on this area; if you look at it in the trailer, for example, it's obviously very much more temporary.

My hope is that at some point we will add a media section where we can put more wallpapers, posters, screenshots and such.  We are all working hard to finish the game, however, so it's not exactly a high priority.

It's been revamped by Alex, who is new on the team, helping us with the huge amount of modeling we still need to do to finish the game. Hopefully we'll get a bio paragraph for him soon on the About page.

The current puzzle count is 590.

The current footstep count is 1756.

P.S.: Last week we showed The Witness at E3. This was the first time we have ever shown the game running on the PlayStation 4. We got some nice write-ups about the game; here are just a few:

Steven Hansen from Destructoid really enjoyed the game!

Tara Long from Revision 3 really enjoyed the game! (She also gave us an award at the show!)

Darren Franich from Entertainment Weekly put us at #1 on the list of most exciting games of E3.

Matt Miller of Game Informer seemed to really like the game, and put together a list of defining features.

Chris Leggett of Gameplanet says The Witness was his personal favorite game of E3. (The folks at Gameplanet who chose the photos to go on the article used some unfortuante, super-old development photos though!)

Tom McShea and Shaun McInnis of Gamespot gave a video summary from the show, seeming to like the game a lot as well!

If you watch carefully and don't blink, and look past all the balloons that are blocking it, you can see the poster (in different proportions) starting around 0:57 in this commercial: http://www.youtube.com/watch?v=QOdW1OuZ1U0

Hey, it is what it is.

Because we picked this one as the final, we have it in a much higher resolution than the other options. If you like it, feel free to use it as a desktop image, or whatever!

... which doesn't look too much different from last time's. Partially this is because we've been working on things that you don't see from this angle.

Here's a shot from a different angle:

We still have so much to do!

On my end, I've been working for the past couple of weeks on UI-related things. For a week I was doing better touch controls (perhaps we'll post a video of this soon), and for a week I was somewhat-rewriting the code that implements tracing lines on panels. This can get surprisingly complicated, and I've rewritten it a few times. I think this is close to the last time, though.

The game tells me the current puzzle count is 574, which is higher than I expected, but hey.

Feel free to use these as desktop images or whatever. Unfortunately we didn't mock these up at high resolution; what you see is all there is!

Most of what we've been working on lately is not visible in this shot, though! The biggest obvious changes are probably in the forest and beach on the left-hand side.

There are some basic hacks going on; for example, right now, we are not drawing the real sky; it's just solid blue. Also, right now we are just loading this starting area, but it should not be long before we can load the whole game (Salvador wrote the streaming system a while back and we have been testing it extensively on the PC, so for the iPad it should Just Work. Famous last words.) Also, as you can see, the panel-tracing controls could use some work.

When we can wander around the island generally, I'll post another video update!

Well, you got a pretty good look at the island in the trailer we recently released in conjunction with the PlayStation 4 announcement. That version has a lot of new modeling compared to the January snapshot, because we were working on the trailer pretty hard. But some new modeling has happened since then, and there are some subtle but nice things to see here.

Since someone is going to ask: The current puzzle count is 499.

Our deal with Sony is a limited-time exclusivity that applies to competing console platforms. Basically it is that you will see the game on the PlayStation 4 for a while before you will see it on the Wii U or the rumored next Microsoft console. (We haven't been disclosed on Microsoft's next console, so I don't know anything about that platform besides the rumors that are on the internet).

This exclusivity does not prevent us from being on the PC or iOS at launch, for example.

People have been speculating that we got paid for PS4 exclusivity, but that is not the case. There was no money involved.

The real situation is that, because we are a small developer, we only have the ability to launch on a small number of platforms at once. We liked the idea of being on a console, and originally we thought we might be on the PS3 or Xbox 360, but eventually we decided not to target either of those, due to the relatively low system specs and the work required to do the ports. After some more time went by, and our release date drifted further into the future, we realized that the next-generation console launch time might be a good time to release the game. (For a while we were hoping to be out substantially earlier than the next consoles, e.g. right now, so we didn't start thinking seriously about this until recently).

To launch a high-spec version of the game on a console, it came down to a choice between the next Microsoft console or the next Sony console. There were people at Sony who really liked the game and were keeping in touch with us about it, and so we naturally started going to their PS4 developer events, got a dev kit, and started playing with it. I don't have good communication with anyone at Microsoft right now, but all our technical people like the PS4 specs a lot more than the leaked Durango specs, and we like the positioning of the PS4 (it's about games) more than what we perceive Microsoft's positioning is going to be.

So we just found ourselves doing a PS4 port. And once we are doing that, it is not practical for us also to think about another console port. We were going to be a de facto console exclusive for Sony no matter what. As I mentioned, there are people at Sony who are very interested in The Witness, so they somehow percolated up through the ranks the idea of showing the game in the launch show.

Of course, Sony wants that show to point out things that are going to be exclusive or special to their console. If we are in their show, but next week we turn around and announce "we changed our mind, we are exclusive on Durango forever," that would be a bit weird. So to be in the show we signed a timed exclusivity for competing consoles. But this was just a formalization of something that was already going to be the case. We like the PS4 and we like the people at Sony we are working with, so it was an easy choice to make the agreement.

Exact timing of our release on launch platforms has yet to be figured out. It just depends on how much we can do and when; we'll know more as we get much closer to the game being done.

(Watch it in HD + fullscreen if you have the bandwidth!)

I didn’t think Witness was going to be an easy project by any stretch, but I certainly didn’t expect it to be one of our biggest creative challenges. I’ve worked on nearly 80 projects spanning feature film, games, ads and other sorts, and my job as sound designer had always been to broaden the experience in every possible way. The Witness inverts this notion.

A tree of any significance in a film or game will have a bird perched on its branches chirping away. Cicadas will buzz. Wind will rustle its leaves. Branches will creak, stress and morph right before your ears. All this happening without you actually seeing it…or really expecting to, interestingly.

In The Witness every element has purpose. A key game mechanic, in fact, relies on the player’s perception as they explore. So, we’re not just talking aesthetics…noise (literally stated) ruins the game’s playability. Noticing, as a buddhist would call it, is key to discovering solutions and even finding the puzzles themselves. Nothing in the Witness is superfluous.

All this said, the island is a tangible living breathing organic place following roughly the same laws of our universe. The trees exist. They have mass and energy. My eyes receive bands of light reflecting off them, and in fact they are quite beautiful. So then what do my ears hear? Silence is not only a bad artistic choice, it's untrue.

At this point, you’ve probably asked, “What about music?” It’s no spoiler that there isn’t a gigantic speaker system in the sky fed by a mic’d orchestra. So….

None the less, the island is a fascinating place, with lots to see, hear and do. Geoff and I ‘the audio team’ have had an interesting time exploring 1000′s of layers of sonic subtly, and finding ways to engage players to listen ever deeper. This is a unique project for game sound, and we though it would be of interest to share our challenges and discoveries since starting with Jon in 2010 (and of course the challenges yet to meet).

Here's the interview.

Click here to view the video on YouTube.

The game's web site.
The Steam page.

The game is primarily the work of one designer, Alexander Bruce, who has been working on it for six years. Players seem to be really into the game, and it's been the top-selling game on Steam for two days so far.

This hiatus was the primary motivation behind writing my previous entry as a cliffhanger: although I had fixed the bug, I didn’t have time to adequately document the bug via debugger inspection, so it would have been hard to write anything meaningful showing how the bug manifested itself in practice. I wanted to wait until I had a chance to go through every last little bit of it, so I could be as specific as possible. In that sense, it was meant to be a bit of a cliffhanger for me as well.

But as it turned out, it was much more of a cliffhanger than I’d originally anticipated.

Good News, Bad News

The good news: congratulations to those of you who posted “floating point precision” as a suggestion for the cause of the bug. You aren’t any dumber than I am.

The bad news: we’re all dumber than we should be.

For those of you who don’t already know all about floating point precision problems, I’ll explain the problem we all thought we were solving. The original code, verbatim, looked like this:

bool ray_vs_plane(Vector3 *xpoint, Vector3 const &p0, Vector3 const &dir,
                  Plane3 const &plane) {
    Vector3 n = plane.get_normal();

    float denom = dot_product(n, dir);
    float epsilon = 0.0001f;
    if (fabs(denom) < epsilon) return false;

    float t = - ((plane.d + dot_product(n, p0)) / denom);
    if (t < 0) return false;

    *xpoint = p0 + dir * t;
    return true;
}

To fix the bug, I wrote a modified version to use for the transform manipulator:

bool ray_vs_plane(Vector3 *xpoint, Vector3 const &p0, Vector3 const &dir,
                  Vector3 const &plane_point, Vector3 const &plane_n)
{
    Vector3 local_p0 = p0 - plane_point;

    float denom = dot_product(plane_n, dir);
    float epsilon = 0.0001f;
    if (fabs(denom) < epsilon) return false;

    float t = - (dot_product(plane_n, local_p0) / denom);

    *xpoint = p0 + dir * t;
    return true;
}

I’ve done exactly the same operation, but instead of using the d value of the plane equation to solve for t, I just move the ray itself to be centered around a point on the plane, so that the d value is implicitly zero. My reasoning was very simple: if I subtract two things that I know are close together (the camera location and the picked point on the manipulator), I will have less cancellation error than if I subtract two things that may be wildly different in scale (the d value and the dot product of the camera location and the plane normal).

This is pretty weak sauce as far as numerical precision improvements go. I am far from an expert on the subject (read some Forman Acton, for example, if you want to see how the pros roll). But, when I did this, it fixed the bug perfectly, so I assumed, at least for the moment, that it must be a reasonable fix.

But, honestly, I didn’t really have a good feeling about it.

Christmas Stew

Sometimes, when there’s a lot to do, you’ll live with a bug fix because it works, rather than because it makes sense. But invariably, at least for me, I end up going back to look more closely at something if I don’t feel like I fully understood it. I am sure this is mostly just an obsessive personality trait, but regardless, I rarely find the time to be wasted, because I almost always learn something from the process.

In this specific case, I didn’t have time to look into the problem in depth before I left. I planned to do so for the eventual writeup, but of course, I couldn’t do much without being able to run the code, and while I was on Christmas break, that wasn’t an option. Unfortunately, this meant that I had the entire Christmas break to stew about the bug that I wasn’t sure I’d really fixed properly.

From the beginning, the thing that didn’t sit right with me about the fix was that when Jon originally reported the problem to me, he didn’t mention anything about it happening more on the outskirts of the island than on the interior. Jon’s a very experienced programmer, so if something like that were happening, he would probably notice and include that in the report, because he’d know it would help me find the bug faster.

Instead, it seemed pretty clear that the bug was related more to the camera angle than the position in the world. While obviously the camera angle affects the input values, and could have a number of effects on the numerical precision of the program, it’s not intuitively the kind of thing that I would expect. If these sorts of algorithms were that sensitive in this way, I would have expected to encounter similar problems with camera-relative plane intersections many times before — but I never had.

The more I thought about that, the more it gnawed at me that I couldn’t go see exactly what was happening and prove to myself why the fix worked. I did have the game on my laptop, because I’d done some initial Linux porting work, so I realized that maybe, if I was lucky, I could reconstruct the problem without actually being able to load and run the editor.

That is when I became very thankful that I had been writing these blog entries, because it meant I had stopped to take this screenshot before I left, specifically so I could do the writeup over break:

Do you see what I see? That lovely entity number up in the corner is pure gold for step one of reconstructing the problem. Because I could see the entity number, I was able to go into the repository and get the data file that says where that entity is placed in the world. That let me know the numbers of the center point of the manipulator: -304.114, 193.4923, 4.7582.

If you’re an experienced 3D programmer, then you probably are thinking the same thing that I was thinking when I saw those: how the hell could I be having numerical precision problems with such low numbers? The Witness’s scale is one unit per meter, which means that, at the scale of the dragging we’re talking about, there should be absolutely no problems with solving a single plane equation, regardless of how sloppily you did it.

But if that was the case, then how did I fix the bug? This lead me to step two of reproducing the problem: writing a test program. I wrote a little C program that would take numbers around that point and, for a large set of programmatically generated planes and rays, perform the intersection using both the old ray intersection routine and my “improved” one.

Perhaps unsurprisingly, these results reinforced my suspicions that there wasn’t enough of a difference between the two routines to account for the original bug. In all the tests, my “improved” routine was no more than one decimal digit more precise; so if the original routine produced intersection points that were actually, say, 0.0001 units away from the plane, the new routine might produce ones that were 0.00001 away.

Clearly, the fix just plain shouldn’t have worked. But it did.

Why?

The Wrong Part of the Post

I would love to be able to say that I guessed the real reason before stepping through the code, but I absolutely didn’t.

When I returned from Christmas break, I eagerly booted up the Witness and stepped through the code very carefully, keeping accurate track of all the numbers as I went, waiting to find that super-elusive numerical thing that I had overlooked. To my complete surprise, it turned out to have absolutely nothing to do with precision.

In fact, somehow, although I didn’t know it at the time, I actually did write about the root cause of the problem in my original post. It just wasn’t in the part that I thought. Indeed, rather than Always Look Behind You foretelling the eventual solution, it was Foreign Languages that contained the hint: due to my naivete with The Witness codebase, I misunderstood what it was doing with planes. It turns out that this one line from the original routine was the entire culprit:

    Vector3 n = plane.get_normal();

You will notice that it only occurs in the buggy version of the ray intersection code. In my version, I couldn’t use the plane structure, because I didn’t want an “a, b, c, d” representation of the plane. I wanted “a, b, c, p”, if you will, where p is an actual 3D point on the plane. So I never called get_normal().

Why does this matter? Well, upon stepping through all the code, I realized that get_normal(), being true to its name, takes the unusual step of normalizing the a, b, c values of the plane before returning them to you as a vector. This means that the a, b, c values of the plane that you pass in when you create the plane aren’t the ones you get out.

That wouldn’t be a problem if it weren’t for the fact that, when you create one of these planes, it does not normalize the values you pass in:

    Vector3 plane_n = cross_product(camera_back, translation_axis);
    plane_n = cross_product(plane_n, translation_axis);
    grabbing_plane = plane_from_point_and_normal(hit_pos, plane_n);

This is my fault on its face, because hey, it does say normal, and normals are traditionally supposed to be normalized. But in keeping with my reference to foreign languages, this is something I really just didn’t think about, because I am used to the way I’ve always done things with planes: nothing is normalized automatically, everything is just left unnormalized, because back in the day, normalization was very expensive and you never did it in a math routine unless it was absolutely necessary.

So plane_from_point_and_normal was actually doing what I thought it would: it just stored plane_n as the a, b, c values, and computed d with the inner product. But get_normal() was not doing what I thought it would. Rather than just returning me the vector {a,b,c}, it was normalizing them before giving them to me, which meant all my calculations with them would no longer agree with the d value computed in plane_from_point_and_normal.

And therein, clearly, lies the entirety of the bug.

Just to underscore how much of a native language issue this really is, when I showed Jon the preview of this article, he said, “If you are thinking of the plane as a geometric object, of course the normal is unit length, what else would it be?? But if you are thinking of it as some specific equation all the time, then it is natural to care about the a, b, c, you put in.” So I can only assume, were he using my math library, he would’ve been tripped up by the fact that I don’t normalize normals automatically, exactly the inverse of the problem I encountered with his.

I honestly don’t believe that this sort of thing is a case of us being presumptuous or sloppy; it’s just the price you have to pay for being able to program complex things at a reasonable speed. If we constantly have to think about what it means to work with a vector, or a plane, or a quaternion, we’re going to be a lot slower. It’s much more efficient to have a consistent perspective, and always rely on it.

And of course, the cost is bugs like this — but that’s a very small cost relative to the benefits.

Partial Credit

So, where does that leave you?

Clearly, automatic partial credit goes to everyone who wrote in suggesting that numerical precision was the problem. You came to the same conclusion I did with the information I knew at the time, which was all you had to go on. You certainly can’t be blamed for not finding the real bug since you would have needed the source code to the Witness math library to even see it. And it is true that the numerical precision of the routine could have been improved, as I believe I was able to verify separately with my test program.

But by the same token, nobody gets automatic full credit. There were a lot of reasons why, even with only the information in the original article, you could have said, “I don’t think it’s numerical precision, Muratori.” Maybe I am ascribing overly heroic attributes to the man, but I feel like Forman Acton would have known, from looking at the plane intersection snippet, that there wasn’t any way to lose enough precision in there to account for the bug as I described it, at least not in any way that would have been substantially improved by the suggested fixes people wrote in (or the fix I actually did, for that matter).

So in order to actually get full credit, you’re going to have to earn it, I think, but on the honor system. Ask yourself, if you had encountered this bug, and you had “fixed” it as I had, would you have gone back and figured out what was really happening before moving on to something else?

If you can honestly answer yes, then you get full credit, for this is about as good as you can hope to do as a working programmer, I think. You’re no Forman Acton, of course. . . but then again, who among us is?

Up until now we have not been in that kind of work mode, because we were figuring out the game, and in that kind of mode, I find schedules to be anti-quality. But once you have everything figured out and need to produce a volume of work, schedules are pro-quality, since they help you gauge how well you are doing, whether you need a reality check on the rate or quality of things you can produce, etc.

Luis has made this handy map that we are using to partition up modeling tasks, which I have then edited to remove spoilers:

I like looking at this map because it reminds me how many distinct areas there are and how they relate to each other.

Today I was showing the game to someone who had not seen it before, and that's always interesting because it helps me see the game through fresh eyes a little. It really feels unlike any other game in the way it is an Open World but also a Small World, where you can be in one location and see several neighboring locations, and that helps keep you oriented, but it also helps you reflect on what you have done and orient on where you are in the course of things you are planning to do.

We still have a ton of work to do, though, and I was reminded of this as well. Not just in detailing all these areas, but in filling out the story, crafting details that further help the areas relate to each other, etc.

It is going to be an interesting time!

Jeff asked me about this part of the iOS App Store's Developer Terms of Service:

We view Apps different than books or songs, which we do not curate. If you want to criticize a religion, write a book. If you want to describe sex, write a book or a song, or create a medical app. It can get complicated, but we have decided to not allow certain kinds of content in the App Store.

And here is what I said:

I think this is the wrong attitude about games, but look, ultimately it is game developers' fault, not Apple's. Apple is treating games as shallow commercial entertainment experiences because they have been taught by game developers that that is what games are.

If we had built a world where games routinely work with serious issues in ways that people care about, Apple would not be able to take this stance, because it would not make any sense.

Why do they say "If you want to criticize a religion, write a book"? It's because it's obvious that banning books is bad, because there have been a lot of books that people find important (and we have had many cultural cycles involving people attempting to ban books, and culture has worked out ultimately that this is not a good thing). Games do not have this history. Right now Apple thinks a game is Angry Birds or maybe Infinity Blade. So who can blame them?

Apple may be badgered enough to change their policy someday, or they may not. But that doesn't matter very much because really it is just a reflection of the general cultural idea about what games are. The only way that idea will change is if a lot of developers make a lot of serious/deep/honest/touching/intrepid games for a long time. I don't know if that will ever happen. How many games can you think of to which you can seriously apply these adjectives? Certainly to none of the top-selling games on the App Store, and certainly certainly to none of those big free-to-play games that are raking in all the cash.

So, game developers are just sort of reaping what they have sown. What else would you expect?

I have not played Endgame: Syria but maybe it is a step in this direction. But to change these attitudes we need a lot of steps, consistently, not just a token step now and again.

Current puzzle count: 505.

The Background

Way back in 2008 when I came upon the core idea of this game, the design was the exciting part. Though I started in games mainly as a technical person who was excited about technical challenges, I have shifted into a mode of thought where, when I program, it's mainly just to get the design done in the most straightforward way.

Since Braid I have viewed programming as mostly pragmatic: I know what I want to do, and I know what kinds of things I have to type in to make it happen, but for the most part, the interesting mental challenges are in design, and programming is rote execution. Once you have enough programming experience, I think 95% of writing a big program is like this. There have been a few times during The Witness when I spent significant effort solving technical problems where I had no idea what the answer would be in advance, like with the terrain manipulation, but even in these cases I treated the technical development in a very pragmatic way: the goal was to get something that is good enough for current needs, not to design a shining jewel of an amazing technical system (back when I was a younger game programmer, I was trying to make amazing technical systems all the time, and it came at the cost of actually getting games done).

Despite the fact that I am trying to focus on design, in order to make this game a reality, I had to start a company, bring people onto the project to build the game, etc. This means that nowadays I am not just wearing the programmer and designer and business development hats, I also wear the producer hat and the (admittedly small at present) HR hat. Maybe if The Witness does well, we can expand the team a little bit so that I don't have to do these things, but for now, that's the reality.

We are very laid back and unstructured in the way we do things, but even so, I feel there's a heavy subconscious load involving running the company, making sure nobody's work is blocked, dealing with money all the time, etc. I often find it hard to do deep design thinking when I am in the office, if only because all these issues are just ambiently around. So, in terms of design, my most productive days tend to be work-from-home days (which for me includes most weekends).

All this up til now is just to explain that if I take a vacation seriously, it can be very relaxing, because that subliminal load is gone, or at least mitigated for a while. The activity of the rest of the company is mostly paused, which removes urgency from my own activities.

At first I didn't know how I would spend vacation; at first I thought maybe I would actually treat it the way most people treat vacation, and not work. Then I thought, hmm, there is a certain sector of the game that I might like to work on, that is definitely very important, and I should get a bunch of that done while everyone is gone, so that when they come back it will be figured out. (I don't want to go into detail about what that section of the game is, because I am avoiding spoilers).

The Vacation

But, inspiration is a funny thing. I worked a little bit on the issues I had planned to work on, but for the first bit of the vacation, up through the 25th, I found myself most motivated by cleaning up old nagging issues on my to-do list, and shipping this improved version of the game to our very few beta players. This involved stuff like improving the feel of tracing lines on the puzzle panels.

A peek at the export script, used to package the game for end-users.

After shipping that beta, I found my attention most strongly drawn toward an unexpected part of the game: an optional area, one that we could completely cut from the game and nobody would know, yet at the same time, it's an area that adds to the overall conceptual arc of the game in a strong way, and would be very rewarding for the small percentage of players who get deeply into the game. (Again, I apologize for the vagueness, but let's avoid spoilers).

I had roughed out a version of this area a long time ago -- maybe a year or two ago -- and had always found the basic idea very exciting. But back then the area wasn't done enough to prove the concept, and over time it decayed while the rest of the game was being worked on (which is natural on a big project like this). But, perhaps because it's an area of the game that nobody else on the team really knows about or understands yet, I had a great motivation to spend vacation time on making this area good. Now that I think about this again it makes a strange kind of sense: when development is pretty much paused, it is somehow appropriate to work on something that nobody else really knows about.

At first I thought the task would just be to make the old stuff work again and then do a straightforward job of the extra functionality I needed to complete the area. Thinking about the programming I would have to do to complete the area, I saw it as a necessary bummer ("I want to be done with this game, but ugh, there is more programming I have to do that isn't even on the mainline to all the things that really need to be done.")

But I started programming that extra functionality, and 10 minutes into it, realized I was not so excited about the design part -- it seemed like a very rote execution, like anyone could do in any game, and The Witness ought to have something better than that. Because there was no pressure, and because I was free to do whatever I wanted, my mind was open enough to instantly see an alternative that would be much truer to the essence of the game. The chance to do this better thing was very exciting. (We'll come back to this subject of being able to see ideas, later).

So I started programming this more-exciting version of the functionality. Then I saw how I could make it even more interesting, and I would start doing that. Then I would see how to make it even more interesting, though that would introduce a lot more programming work. If I were feeling the regular pressure of the game schedule, at this time years in when it is time to be finishing the game, I might have said, "naah, we don't have time add that extra work". But because it was vacation the work was free, so I said screw it.

I worked in a fiercely productive way for a couple of days. New ideas would come. They would raise new issues. I would program some things, and in that way that programming often goes, new problems would come to light that I hadn't thought of, and those problems introduced even smaller 2nd-order problems when confronted. But by this time I was fully energized and just had dogged determination and was fully enjoying the whole process. In the end I programmed at least 5 times as much functionality as I had planned, but rather than being bummed about it, I enjoyed the whole excursion.

Those who know me will tell you that I am not predisposed toward painting things in an unduly positive light -- but I will tell you that these past four days or so have been amazing and wonderful. Here's a shot from what I did (the visuals are 'programmer art', as usual):

It is hard to tell you why this is so cool without huge spoilers.

It's so amazing and wonderful because

it's been a long time since I worked this hard and this effectively. Keeping in mind that memory is sometimes not as reliable as we think, I would tell you that the last time I worked this hard and this well was when starting my first, ultimately doomed, company (with a friend from college and a 3D artist we recruited from that same school and unfortunately put through years of meager-wage toil).

Here are some shots from the first professional game I ever worked on, back before we had a concept of 'indie development'. Software rendered, subpixel-accurate texture mapping, assembly-language inner loop that commandeers ebp in order to utilize a precious extra register, a la Mike Abrash, and oh yeah, it was multiplayer-only, 32-player, over 9600 baud modems:

We were working on this game from 1996 to 1999 or so. 1996 was the year I got into professional game development, and in hindsight, it appears to have been the hardest possible time in the history of video games. Indies of today, y'all have no idea how much easier it is to make games now than it was in 1996. (I hate to be an old fogey who's all "back in my day, we had to walk to school in the snow, barefoot, uphill both ways," but in this case it is absolutely true.)

That game was never very successful. It had design problems, but also, we didn't really know how to talk about or promote a game; lastly, this was before many people played games over the internet, so the potential audience was pretty small. (Trivia Tidbit: There was another online game being supported by the same publishing service as our game, and this other game was more popular, despite being relatively simple in a technical sense, which annoyed the hell out of us on a subliminal level that we wouldn't totally admit. The guys making this game were John Vechey and Brian Fiete; our producer at the publisher was Jason Kapalka, who later went off with these guys to start a company called PopCap, which last I heard had done okay.)

Back in those days I worked very hard. When we started the company I knew nothing about building 3D games; we learned very quickly. I would long hours, late into the night, intensively. At some point we all burned ourselves out, I think.

After we closed down that company, I went on to do a number of contractor things, game industry lecture things, magazine article things, and working on various independent projects that I never finished. Through that time, I was still very burned out. If you have never experienced burnout, it's hard to explain. Burnout is not just "I don't feel like working right now"; it is about your mind refusing to permit you to work, because it has seen what happens when it lets you work.

Eventually, I swore that I would design a very good game and finish it and release it to people, and the result is a game that was successful and that many people like:

A main part of the reason this game exists was that I instilled a certain discipline in myself: I wasn't going to do this game because I was excited about a technical thing, or excited about programming; I was going to minimize the amount of programming, firstly so that the game would get finished at all (I had plenty of experience by this point at not-finishing-projects, as did many of my friends; it was an endemic problem), but also so that I could focus appropriately on the design and build something very special.

Despite the fact that I kept the game technically simple, there was still a huge amount of programming to do. But thankfully, the strategy worked and the game was completed. Eventually.

Even though I'd made a successful game, I was still burned out to some degree. In 1996-1997 I was working at full capacity and was very productive, at least inasmuch as one might expect for a programmer of that age who was first starting to get real-world large-project experience. But after being burned out at that time, I never again worked at full capacity.

What I am getting at

I am saying all these things just as a way to get around to explaining that, during this Christmas vacation, I finally recovered fully from burnout. For about 3 days, I was working at 100% capacity, which I have not done in 15 years. It felt wonderful to have that back, to not succumb to the story that maybe game design and programming are like math of physics where you peak at 25. These few days were way better than when I was 25, because now I am substantially better at programming, and I am good at design also, and games are so much easier to make than they used to be; so the design cylinders are firing and then they turn it over to the programming cylinders, etc, and very quickly a lot of high-quality work gets banged out. And it's not just arbitrary good work, but work that helps to round out the game that I already feel is the best thing I have ever worked on, that helps put grace notes in just the right places, in the manner of exploratory game design that Marc ten Bosch and I spoke about at Indiecade 2011. So this period of intensive work was seasoned and tempered in a way that was not possible when I was 25.

It has been a wonderful time and I look forward to doing more of it. Part of what allowed it to happen this way was that lack of pressure that I have mentioned, due to being on vacation. When a design idea would come to mind, and I would see that it might be cool but would lead to a lot of work, well, this week I was able to let it happen, and I was able to enjoy the challenge of going off on a minor programming rathole in order to do something really good in the design. This is to some degree a reversal of the discipline I instilled to get Braid done; you can think of it as a correction term to the equation. The 0th order term is knowing how to program and design; the 1st order term is having discipline to be able to execute efficiently without being carried away with ideas that will scuttle production; but the 2nd order term is that it is joyful and interesting to have some wiggle room in there, sometimes eroding away a bit of the 1st order term.

There is a kind of open curiosity about programming that I haven't had in a long time, that I allowed myself to re-discover this week in the freedom of exploring that 2nd-order term.

(Richard Feynman has an anecdote about how he was burned out on physics, and how he started to get over it, which I think bears some similarity).

I am not explaining any of this very well. But I am doing the best work of my life, right now, and it feels great, and for the first time I could see myself working in this same fashion for the rest of my life and feeling happy that I am operating at something like my full potential. So there you have it.

I mentioned earlier that I would talk further about the ability to see ideas, but I guess I was lying, because this is already too long and rambling. I'll save that for a future post.

It's worth pointing out that I started Braid while working on what was supposed to be a vacation. And back when I got the impulse to start the Experimental Gameplay Workshop, I was on a plane back from a vacation to Africa. So apparently vacations are good times for me, even if I don't treat them the way Normal People do.

I do not know who made this rock or how long it has been there, but when I first happened upon it, I found that it resides in a nicely isolated part of the island where there’s nothing else in view, so the frame rate is always high even in full debug mode. Plus, it’s big, unobstructed, and easily pickable in the editor without fear of selecting any other objects. These properties make it perfect for testing editing features in progress, so the rock and I have become fast friends.

Normally, I do not want large rocks to move around. If I’m in the game engine, and a giant rock starts moving, I know I’m in for some debugging work. But if I’m in the editor, and I tell a rock to move, I expect it to move. That’s the whole point of having an editor. So when this cliffside rock, which otherwise had been so pleasant, decided that it would only move in certain directions, I knew I had a bit of a mystery on my hands, one that eventually forced my brain to remember some very important things it had forgotten in the ten years since I last did any serious 3D graphics programming.

Foreign Languages

Working in someone else’s codebase is much like moving to a foreign country that happens to use the same alphabet as you do, but not the same language: technically, you can read, but you don’t really have any idea what most things mean. At first, every little thing you do is a painstaking linguistic procedure, and you have to constantly check and recheck things to make sure you actually know what’s going on. Over time, as you gain more experience, you start to intuit the way things work, and you’re able to operate at closer to your full speed.

I’m used to the process of working in unfamiliar codebases since I’ve worked with so many of them over the years. I tend to approach each new one the same way: first, I write something very isolated, and keep it walled off. This lets me start making some contributions right away, while I gradually learn how to interface with the rest of the code. In the case of The Witness, this was Walk Monster. Next, I try to do some performance optimization or bug fixing, so I learn to work locally, but directly, with existing code. That was fixing the five-second stall. Finally, I start picking real features to add to other people’s code, preferably in the systems that I will most likely need to learn well.

Since I’m working on the collision system, and at this point I am certain there will need to be some interactive tools for it, I picked the Witness editor as the best place to start adding real features. After playing with it for a while, I selected two features I thought would be good additions from both the standpoint of learning the code, and for be generally useful for people working on the game. The first feature was compatible camera controls, and the second was a translation manipulator.

By “compatible camera controls”, I mean camera controls that work like common 3D art packages. My understanding was that the Witness team did their modeling in Maya, so I put a toggle in the editor that, when enabled, allowed you to use ALT-drag camera movements that replicate the way Maya moves the camera. After checking this in, I learned from the artists that some of them prefer 3D Studio MAX, so I added another button that emulates its camera controls (once you do one, it’s very little incremental cost to add others, as they’re really all just other sets of bindings for the same movements). All of this went rather smoothly.

Unfortunately the second feature, the translation manipulator, did not go so smoothly at all.

The Translation Manipulator

By “translation manipulator”, I mean the standard three-axis overlay that modern 3D packages have where you can click and drag on any axis and move your selection along that axis. Depending on the package, there are also little squares on each plane that allow planar movement. I chose to include these, but to allow them to be toggled separately since not all packages have them and I didn’t want them to be in the way if an artist wasn’t accustomed to using them.

Compared to the camera controls, it was moderately more work to understand all the intricacies of the editor code involved with translation. But I worked my way through it, and eventually I had a translation manipulator working reasonably well in my test world, which was a completely empty world with just a few boxy entities that I could move around to see if everything was working.

When I was confident everything was solid — undo was working, occlusion was working, etc. — I tried it out on the full Witness island for a bit. Nothing seemed immediately wrong, so I checked in the feature on a toggle switch that defaulted to “off”, and asked Jon to check it out when he got a chance. I figured it was better not to check it in enabled by default and have the artists all start using it, only to find out it had some critical bug that would corrupt things in some irreparable way, losing lots of valuable work in the process. Since Jon does a lot of work in the editor, and he wrote a lot of the editor code originally, I figured he would know the things to try that might break so we could fix them before any real damage was done.

When I heard back from Jon, he said it did seem to be working correctly, but sometimes “moving along the Z axis wasn’t working”.

The Occasionally Movable Rock

I was not at all surprised to hear that Z-axis translation was unreliable. This was something I was expecting to have happen. Surely something was broken in the way my code interacted with the existing planar movement constraints, which were buttons the editor had always provided and which were the only way of constraining interactive movement before I added the translation manipulator. But I checked the code thoroughly, and I just didn’t see any way that I could be mishandling the Z axis in the intermittent way Jon had reported.

I tried playing around with movement more in the editor, to see if I could reproduce the problem myself. Soon, I started to notice that when I played with the solitary rock in the ocean, it wasn’t really “sticking” to the cursor as well as it should be; the code I had written was supposed to feel as if you had “grabbed” the manipulator, and it should follow your cursor rather exactly. It wasn’t doing that.

Then I noticed something even stranger. Depending on how I rotated the camera, I was able to find angles at which Y-axis dragging moved the rock extremely slowly even for large cursor movements. At one particular camera angle, the rock refused to move along the Y axis at all! For example, at this angle, I could move the rock along the Y (green) axis, albeit slowly:

and at this angle, I could not move the rock along the Y axis at all:

even though the other two axes worked fine, and from other camera angles the Y axis worked fine, too. Since the manipulator supported entity-relative axial movement as well as world-relative axial movement, I tried switching to entity-relative axes to test if there was something wrong with the Y axis specifically. I rotated the rock such that the Z (blue) axis was pointing in the direction that the Y axis pointed previously:

To my surprise, it now refused to move along Z! So it really had nothing to do with which semantic axis you were moving, it seemed to be the direction in the world and the angle of the camera alone that mattered for reproducing the bug.

Looking for more ideas about what the bug could be, I tried loading the empty test world again and playing around with the manipulator there. Again, to my surprise, it was completely impossible to reproduce the bug. No matter what camera angle I used, or where the axes were pointing, the always dragged perfectly, and seemingly at the correct speed to boot.

What on earth was going on? Clearly, it was time to think more deeply about the code and the behavior it was exhibiting.

Ray Versus Plane

In most 3D editing operations, you must confront the problem of turning a 2D cursor position into something useful in three dimensions. Since a 2D point on the screen mathematically represents a ray in three dimensions (everything in the world that is “under” that point from the perspective of the user), this typically involves taking the ray and intersecting it with something. For example, to implement picking of objects in 3D, you typically intersect this 3D ray with the objects in the world, and you see which intersection is closest. That will be the closest object under the cursor when the user looks at the screen, and therefore the one that they most often expect to pick when they press the mouse button.

My translation manipulator also works by intersecting the ray implied by the cursor. Knowing which axis the user clicks on is analogous to the picking problem, and I essentially implemented it that way: I pretend that there are three long, skinny boxes that align with the axis arrows, and I test to see if the cursor ray intersects them.

Picking the axis is straightforward, but things get more complicated once dragging begins. Although I know which axis the user started dragging, in order to actually move things properly, I have to have a way of computing exactly how far they have dragged the object along the selected axis. Intuitively you might think well, why not just extend the box used to pick the axis to be infinitely long along the dragging axis, and continue intersecting with it as the user moves the cursor? That’s not a bad idea, and it would work, but it isn’t the best solution.

The reason is because it doesn’t feel very good to use. Although it will feel fine if the user drags right along the axis in question, a lot of times users don’t do that. They tend to drag erratically, and pay more attention to the object’s position than how closely they follow the line implied by the axis. This quickly moves the cursor off the imaginary box entirely, so the ray implied by the cursor won’t intersect it. This causes the manipulator to stop working until the user moves the cursor back onto the axis, resulting in a sticky and unpleasant user interaction.

What we’d rather have happen in this case is for the dragging to happen as if the user had moved the mouse to the point on the axis closest to where they actually dragged, even when they’ve wandered quite far from the axis itself. To see how to solve this problem geometrically, imagine a user dragging the mouse along the X axis of a 2D plane. If the user starts dragging erratically, and moves along Y as well, their cursor will end up out in the middle of the plane somewhere. Figuring out where they would have been on the X axis is quite simple, though: ignore the Y coordinate, and just use the X coordinate. Nothing could be simpler.

But how do we do this operation for an arbitrary 3D axis and a ray extending from the cursor? It turns out you can do it exactly the same way. All you have to do is pick a 3D plane that happens to contain the dragging axis, then intersect the cursor ray with that plane. That gives you a nice 2D coordinate on a 2D plane, where the dragging axis is analogous to the X axis in the simple example. You then “throw out the Y”, and are left with a drag that always feels nice in 3D, sticks the object tightly to the cursor, and does not require the user to drag close to the axis.

You don’t need any fancy math to do this. It’s entirely constructible with simple operations available in any math library: dot product, cross product, and ray-intersects-plane, all three of which were already available in The Witness codebase.

So my translation manipulator was built entirely out of the most basic 3D operations in any math library. Yet somehow it was exhibiting very odd behavior atypical for a simple system. How could this be? What was I missing?

Debug Rendering

My first thought was that I must be picking the dragging plane poorly. Constructing planes in 3D with only partial information is always a little dicey; you need three non-collinear points to define a plane, but often you want to construct one with only one or two points. The translation manipulator is a perfect example: it needs a plane to operate, but it only really has a line to work with, which is equivalent to only two points. There are, in fact, an infinite number of planes that pass through the dragging axis of the translation manipulator. It’s much like a waterwheel: the dragging axis is the center axle, and each blade of the wheel is another valid choice for a plane.

So how do you pick one? The best method I’ve found for translation manipulators is to use the location of the user. Since we want the user to be able to drag around on this plane easily, it makes sense to say that, of all the planes containing the dragging axis, the best one is the one that is most visible to them. This avoids the potentially bad cases you can hit if you pick something independent of the user’s location, such as the resulting plane being edge-on or nearly edge-on so that the user cannot actually drag along it effectively.

Since even simple 3D math like this can be difficult to verify by inspection alone, my first inclination was to add some debugging rendering to verify that I was picking the dragging plane properly. Since the camera angle clearly mattered, and the plane was one of the only things dependent on that angle, it seemed like a likely culprit. So I tried drawing the normal to the plane in yellow at the base of the manipulator:

Even in the cases where the dragging failed, the plane normal looked right. Maybe the positioning of the plane was wrong? I tried computing the location of the plane directly, by moving along the normal by the plane constant d and drawing the normal there. This, of course, put the normal nowhere near the manipulator, so I had to look around for it (the plane equation gives you the point closest to the origin, which has no real relationship to where the manipulator might be, other than that they both lie on the same infinite plane). But once I found it, even though it was far away, looking back toward the manipulator it still did seem like it was the right plane.

Next I moved onto the intersections. I tried drawing the intersection point that I had computed when picking the axis in the first place, and also the point that I used to compute the plane constant in the first place. Both looked correct.

But then I tried drawing the actual intersection computed at each drag update, and that didn’t look correct. In fact, it didn’t really “look” at all. Although I was specifically telling the renderer to draw a white sphere where I’d found the intersection of the cursor line with the plane, no white sphere appeared. I checked the code again, and it looked correct, so I tried doing a few more test drags, this time from angles that had seemed to be working before. Now, the sphere showed up, although it was bigger than I thought it should be, which was odd.

I tried rotating the camera. I wrote the code so as to leave the sphere rendering wherever the last dragging intersection occurred, so if I stopped dragging, it just stayed where it was. This allowed me to rotate the camera and see precisely where the sphere was located, and as I did, I realized that the reason the sphere appeared too large was because it wasn’t even close to being on the plane I thought I was using. It was much closer to the camera. Weirder still was that, at the buggier camera angles, the sphere was jumping around erratically, and wasn’t really sticking to the cursor, which it always should do no matter what plane was being used. I was, after all, intersecting the cursor line with a plane. How could the intersection point possibly lie anywhere but directly under the cursor, regardless of the parameters of the plane?

It was at that moment that my math brain finally woke up. It had been sleeping for about a decade, which is how long it’d been since I’d done any real mathematical programming. In an instant, I immediately knew both what the problem was and how to fix it.

Always Look Behind You

The problem was embarrassingly obvious when you consider the clues I’d been generously given and yet completely ignored. Remember when I said I drew the plane normal at the location indicated by the plane constant, and I had to look “far away” at the manipulator to see if it looked like it was in line with the plane? Well if the generally erratic behavior of the manipulator hadn’t been telling enough, that little episode should have been the dead giveaway.

At this point, every serious 3D programmer who hasn’t been doing other work for ten years surely knows what the bug was. But for the sake of people who are less experienced, I decided to end this article with a bit of a cliffhanger.

I am going to tell you that you can find the bug by looking at this simplified version of the ray_vs_plane function in The Witness that I was calling (the real version does some epsilon checking in case the ray and plane are parallel, but otherwise it is identical):

Vector3 ray_vs_plane(Vector3 ray_origin, Vector3 ray_direction,
                     Vector3 plane_normal, float plane_d)
{
    float denominator = dot_product(plane_normal, ray_direction);
    float numerator = dot_product(plane_normal, ray_origin);
    float t = -((plane_d + numerator) / denominator);
    return ray_origin + t*ray_direction;
}

And I will also show you this screenshot of what it looks like if you turn around 180 degrees from the rock and look back at the island itself:

But, until next time, I’m not going to tell you what the bug actually is. Can you find it, and figure out how to fix it, before then?

Granted, it is not uniformly horrific. It ranges from only slightly scarey (xinput, comdlg32) to full psycho (DirectShow, Event Tracing, TabletPC, etc.). But no matter which part of the spectrum you encounter, programming on Windows tends to be about doing a lot of unnecessary research, experimentation, and debugging due to a confluence of poorly designed APIs.

Since time spent debugging Windows issues isn’t often chronicled, I thought I’d go ahead and describe my experience this Saturday of tracking down a Windows bug in The Witness.

The Symptom

Apparently, not everyone on The Witness team was seeing the symptom. But trust me, it was there.

On both Jon and my machines, there was a period during the initial loading of the game where you couldn’t do things like ALT-TAB back to the debugger. If you set a breakpoint then ran the game, when the breakpoint eventually hit and the debugger came to the front, it would wait about five seconds before it allowed you to actually interact with it. Sometimes, in rare cases, it would go into some sort of psycho mode whereby mouse movements stuttered, and the computer would literally beep at you as the mouse moved, for several seconds. Then everything would return to normal.

While not technically a showstopper, this is the kind of bug that’s worth fixing if you can, because it costs you real development time every day. Five seconds wasted every breakpoint is bad news, and people often underestimate the effects of frustration that can build up over time due to flaky and unpleasant development environments. So I decided to try to track it down.

Because Windows is a closed-source platform, there’s no great way to track down problems of this nature. However, having programmed on Windows for over twenty years now, I have a nose for it, so I felt like I knew the best place to start: since there was definitely a UI blackout somewhere during startup, there had to be something occurring early in the execution of the program that was causing at least that particular symptom. So before doing anything else, I started stepping through the entire startup sequence of the program to see if anything out of the ordinary struck me.

I very quickly came upon this code:

// @@ The hook appears to install properly, but is never invoked :(
keyboard_hook = SetWindowsHookEx(WH_KEYBOARD_LL, LowLevelKeyboardProc,
                                 GetModuleHandle(NULL), 0);

Now, I don’t want to make it sound like the “find the source of the problem” phase of Windows debugging is straightforward. It usually takes a very long time, sometimes multiple days. But as luck would have it, this was literally the first piece of code I suspected, and it turned out to be the culprit.

I commented out the SetWindowsHookEx, ran the program, and poof! All the symptoms were gone.

The Problem

From experience, I’ve learned that it’s always best to fully understand a problem before you fix it. If you just patch over its symptoms but never figure out what the problem really was, it will often come back to haunt you.

In this context, that meant two things: first, I should figure out specifically why the keyboard hook was causing the UI stalls; second, I should determine why the code had been trying to call SetWindowsHookEx in the first place, even though apparently it never quite worked (according to the “is never invoked” comment).

The first part turns out to be obvious in hindsight, but it took me a little while to put all the pieces together and be sure I had a solid explanation for what we were seeing. As with any puzzle, looking at the finished picture is easy, but figuring out what that picture might be when all you have are little pieces coming in one at a time is much more difficult. Here’s the best picture I could come up with, but of course there’s no way to completely verify it external to Microsoft:

Normal Windows hooks are pieces of code which execute in the address space of the eventual message recipient, not the installer of the hook. It is for this reason that, normally, if you want to install a global hook, you must put the hook code in a DLL, because that DLL must be mapped into the address space of every running executable on the machine that can receive Windows messages.

But low-level hooks are completely different. A low-level hook like WH_KEYBOARD_LL is a piece of code that resides in only the process that registers the hook. Windows remembers which thread registered the hook, and when any other executable is about to have a hooked message posted to its message queue, Windows actually waits until it can switch contexts back to the hook thread, run the hook in that context, and then finally deliver the message.

This leads directly to the symptom that we were experiencing. When the game is loading, its main thread isn’t yielding to Windows frequently like it is during normal play. Since the main thread is the one that registered the hook, Windows must wait for that thread to finish all the load-time work it was doing and call a function which yields to Windows in order for Windows to actually call the hook procedure. Since no keyboard processing can occur until the hook procedure has run, the keyboard becomes unresponsive as messages back up waiting to be processed through the hook. This completely explains the UI problems during startup, and fits with the oddity that the mouse never seemed to be affected, just the keyboard.

When the game hit a breakpoint, the keyboard hook couldn’t execute at all because the debugger had halted the thread required to execute it. This would have meant that keyboard input would be completely halted, and the debugger would never be able to receive keystrokes, if it weren’t for the fact that Windows actually uses a timeout on hook calls. If the timeout value is exceeded waiting for the hook thread to become available, the hooking executable is assumed to have crashed, and it will silently remove that hook and continue with normal operation.

That timeout value? Five seconds — right in line with our observed pause (technically, it’s whatever is set in the registry under HKEY_LOCAL_MACHINE\Control Panel\Desktop\LowLevelHooksTimeout, but the default is five seconds).

So, no mystery as to the pathology. But what about the second part? What was the code actually trying to do?

The Windows Logo Keys

Although SetWindowsHookEx might seem like an odd call to make in a game executable, it’s actually quite common. It’s there to prevent the Windows logo keys on modern keyboards from ruining full-screen games. Because the keys are placed in a location that’s easy to hit by mistake, many gamers would accidentally hit them, causing their games to be instantly deactivated by Windows in favor of bringing up the start menu. So widely acknowledged was this problem that, when these keys were first introduced, Microsoft itself published a recommended work-around. From Disabling Shortcut Keys in Games:

This article describes how to temporarily disable keyboard shortcuts in Microsoft Windows to prevent disruption of game play for full screen games... Use a low-level keyboard hook to filter out the Windows key from being processed.

The article goes on to show sample code that is basically the same code that was being used in The Witness. It’s a simple SetWindowsHookEx call with WH_KEYBOARD_LL whose hook routine does nothing but block VK_LWIN and VK_RWIN keys from being processed. The only difference between the code in The Witness and the code in the sample is that The Witness would conscientiously remove the hook when it no longer had focus, and then reinstall the hook when focus was regained. The sample code, by contrast, just set a global variable.

So if this was what the code was doing, why the frowny face? If you look back at the comment, it states very clearly that the hook was never being invoked, even though no error message came back from Windows. And, although I normally remap my Windows keys via the registry, I went ahead and mapped them back to see if The Witness was properly blocking their effects.

Much as the frowny face foretold, it wasn’t.

Obeying the Static Discipline

I don’t know anything about digital circuit design, but I once watched a lecture by Gill Pratt where he explained something called “the static discipline”. In essence, it is the requirement that any component in a circuit must accept voltages with a certain range for “1” and “0”, and then must output its own results in a certain range. These ranges are defined so as to require that components always produce equal or better conditioned voltages than their inputs, thus ensuring that the digital signals being propagated through the circuit don’t degrade into noise due to lots of tiny losses.

For some reason, I took this concept to heart in a programming sense and have found it is a good rule to code by. My version of the static discipline, adapted for software, is that whenever you are making a modification to a piece of code, you should always leave it in a state of stability equal to or better than how you found it. And preferably the latter.

All too often, people go in to fix a problem or add a feature to a piece of code, and they just hammer on it until it does that one new thing. The resulting code is then usually more fragile, less well designed, more unnecessarily complex, etc. To try to prevent myself from having this effect, I try to observe the static discipline.

And so it was with this Windows key situation. I could see what the code was supposed to be doing, and although I could reasonably fix my bug by just commenting out the hook (because it didn’t work), I felt it was the more disciplined thing to do to figure out why the hook wasn’t working, and to implement something that did what it was trying to do. Since there was no obvious bug, I had to start experimenting.

After adding some debug outputs and running Spy to keep an eye on the window messages, the first thing I tried was removing the part of the code that unregistered the hook when his window lost focus. Since I didn’t know exactly how his windowing code worked at all levels, I figured it wasn’t entirely out of the question that it could be getting disabled prematurely.

Although that turned out not to be the case, this experiment did pay off unexpectedly (as many do): although the keyboard hook still never got called when The Witness window had focus, the hook did start getting called when it lost focus. Yes, oddly enough, the hook was working just fine for everyone else’s window, but on the one window where it actually needed to work, it wasn’t working.

This lead to the obvious question, what was The Witness doing differently?

Raw Input

This was a case where having lots of Windows programming experience probably saved a lot of time. Instead of having to experiment blindly, I knew right off the bat that Windows Raw Input probably had something to do with the situation. I’ve debugged lots of Windows front end code in the past, and any time Raw Input, DirectInput, XInput, or anything else with the word “input” in it is involved, you know that things are going to be going a little haywire.

I commented out the device registration call for Raw Input, and lo and behold, the hook started working for The Witness window, no problem. So, although I was completely unable to find any documentation on MSDN that discussed what was going on under the hood, it must be the case that Raw Input processing happens before low-level keyboard hook processing in such as way as to prevent the keyboard hook processing from ever seeing keyboard messages on a Raw Input-registered window. Normally, this wouldn’t be a problem, but in our case, the Microsoft-recommended fix for the Windows logo key problem relies on keyboard hooks. The Witness needs Raw Input to get good low-level relative mouse movements, but it also needs to disable the Windows logo keys; sadly, this is the kind of situation that is all too routine for anyone who programs Windows professionally.

I couldn’t think of any plausible ways that I could convince Raw Input to start using the Windows keyboard hook, since I don’t have access to any of the Windows source code and I have no idea how it works internally. So I focused my efforts on Raw Input.

I experimented with just about everything I could think of. I tried not calling DefWindowProc on WM_INPUT calls, to see if it was the default window processing that was using the Windows logo key events to switch to the start menu. No such luck. I tried using RIDEV_NOLEGACY during Raw Input initialization to see if preventing keyboard messages from being generated would solve the problem. It didn’t. After many failed attempts, and finding no relevant information on the web, I decided to call it a night and sleep on it.

When I woke the next morning, for some reason I remembered the most important thing anyone programming Windows must remember: never, ever trust the documentation. It’s always either inaccurate or incomplete. So when I sat down at my keyboard, the first thing I did was try using the RIDEV_NOHOTKEYS flag when initializing Raw Input.

You see, the previous day I hadn’t bothered to try this flag, because it explicitly says in the documentation that it does not block system-level hotkeys, only application-level hotkeys (like the kind you create yourself with RegisterHotKey). But why was I believing that? That was just something somebody wrote down when they needed to create a documentation page. They probably never even saw the actual code for Raw Input in the first place.

Surprise surprise, RIDEV_NOHOTKEYS worked. After all the fussing, all that had to be done to make The Witness impervious to Windows Logo keys was one little flag during Raw Input initialization. No keyboard hook necessary.

Zero-Sum Game

It’s worth taking a step back for a moment and asking the question, why did all of this happen? Why did I have to spend several hours working on code that has nothing to do with my project, is not valuable in any intrinsic way, and whose specifics are relevant only in a fleeting temporal sense (there will be no need to know anything about RIDEV_NOHOTKEYS when the next user input model for Windows is invariably introduced)?

The answer is the same on Windows as it is on most every other platform that exists today: the platform providers just don’t spend real resources designing and improving their APIs. If there had simply been an API call, WindowIgnoresWindowsLogoKeys(HWND), then everyone would always just call that, and the platform developers and Microsoft could field it and do the right thing, no matter how many times they revised their platform and its infrastructure.

Better yet, way back when they first started catering to games with Windows 95, they could have introduce a call like BeginFullScreenGame(), and that could have done literally everything for you: initialize the window, set up the input stream, prepare the sound device, etc. Then they wouldn’t have had to ever worry about the Windows logo keys at all, because they would have been automatically disabled in any application that initialized itself with that call.

But instead of any of these things, Microsoft chose to ship a new feature (the Windows logo hotkey) without ever thinking about the consequences. Then they patched over its problems by telling developers to install a keyboard hook (SetWindowsHookEx), something that has nothing to do with the actual problem but which was the most expedient way for them to get the problem solved without doing any real work on their platform. Then they broke that method, which they’d previously advocated, when they introduce Raw Input, so (presumably) they introduced a new way to do the same thing (RIDEV_NOHOTKEYS), although they failed to document it. Then they didn’t even bother to publish an obvious tech article explaining this change, or even revise their old documents saying that SetWindowsHookEx was the preferred solution.

As broken as all that sounds, it is absolutely par for the course. On every project, in every discipline, developers just like me spend hundreds of hours doing nothing but diagnosing and repairing problems with the platforms on which we ship our software. It may sound like it’s just “part of the job”, but the pernicious thing about it is that development resources are always finite. External to the platform, development is always a zero-sum game. The hours I spent tracking down this bug were hours that won’t get spent making parts of The Witness better, and there’s nothing me or the team can do about that. If we want to ship a polished project, it’s a sacrifice we have to make.

Despite not even being finished, The Witness is already one of my all-time favorite games. I’ve been playing it since Jon started developing it several years ago, and I could not be more excited about its upcoming release.

Unlike Jon’s previous game Braid, the asset and programming scale for The Witness is much further toward the “triple-A” side of the scale than the “indie game” side. As anyone who works on these types of projects knows, the work required to successfully pull off a game goes up dramatically when you go that direction. The Witness already has several more people working on it than Braid did at its peak, but as with any project of this size, there are still many parts of the project that need more attention than the principals can afford to give them.

For this reason, it was always my intention to try to find some time to help out on The Witness when it came time to ship it. So this past Thanksgiving, Jon and I sat down and he went through a list of things in the codebase that he thought would benefit from the attention of an extra programmer. After considering the relative importance of the items, we decided that the player movement code was the place where improvements could have the biggest impact on the player’s experience.

Walkmonster in Wall

In the context of The Witness, the goal of player movement code is to be as unobtrusive as possible. The player is supposed to fully immerse themselves in an alternate reality, and every last detail is important to this experience. The last thing you ever want to happen is to have the player notice the fact that they’re sitting at a computer asking it to move a virtual viewpoint around.

For that reason, the player movement code has to be rock solid. If you can get stuck on edges, trapped in walls, fall through the floor, walk through a cliff, walk down a hill but not back up it, etc., it immediately breaks the immersion and reminds you that you are playing an artificial experience mitigated by an unreliable system. It can also lead to disastrous consequences for the player in some circumstances if there’s no way for them to recover from the problem with restarting or reloading a (possibly very old) save game. If you play games often, you’ve probably encountered one or more of these problems in shipping titles, and you know what I’m talking about.

Last week I began to work on this problem. I decided the first thing to do was to write a few integrated tools for working with the player movement code so we could analyze it and see how it was currently behaving. Immediately, when I opened up the project, I found myself with a familiar conundrum: what to name my first source file. This is always the most important part of any project (as Bob Pollard once said of band names and album names). If you have the right name for your source file, it’s smooth sailing from there. Pick the wrong name, and you might as well call the project off.

But what to call a system for bulletproofing player movement code? I’d never been asked to write code for that before. When I thought about it, I realized that I’d only personally seen evidence of similar code once before: when playing an early beta of Quake. There were some bugs with the monster placement, and in the console window, you could see error messages announcing the fact that the monsters, instead of spawning on the ground, had spawned partially intersecting with the level geometry. Each debug message began with the phrase “walkmonster in wall at. . .”

Bingo. It’s hard to ask for a better source file name than “walk_monster.cpp”. I was pretty sure, from that point forward, the code was going to come along nicely.

Drive Toward Point

When you’re trying to test a system, the most important thing to do is actually test the system. Even though that sounds like a straightforward rule, people writing tests often fail to observe it.

In this specific case, there are very easy ways to think that you were testing the player movement code without actually testing it. One example would be to do some kind of analysis on the collision volumes and walkable surfaces in the game, looking for small areas, gaps, etc. Once you’d eliminated all of these, you’d then proclaim the world safe to traverse and move on.

But this is testing the data, not the actual code. It’s still easily possible to have bugs in the movement code that result in bad behavior even with sanitized data.

To avoid this kind of trap, I wanted the testing system to remain as close as possible to what a human does to actually control player movement in the game. I started out by writing two routines to serve as the building blocks of this kind of testing.

The first is the closest to real human usage. It’s an update call that hooks into The Witness’s input processing and feeds it synthetic mouse and keyboard events. It can do some basic things that humans might do, such as looking around, walking toward a point, looking toward a point, and so on. It does these through nothing but emulation of human interaction with the mouse and keyboard, so I know that everything in The Witness’s input path will be running as it actually would during testing. I’ll talk more about this system and how it’s used in later installments.

The second piece of code is one step removed from that level. It’s a function called DriveTowardPoint that takes two points in the world and, calling the existing player collision system, attempts to move cleanly from one to the other. When it returns, it provides information about the attempt such as obstacles encountered and whether or not the destination was successfully reached.

This function is not quite as reliable as the synthetic input method of testing, because it eliminates a portion of the actual player movement system from testing. For example, any erroneous state associated with where the player is that the collision system might have built up won’t affect tests using this function. Nevertheless, I felt it valuable to have this level of testing available because it can much more rapidly test large areas, since it does not require the entire game loop to run, and thus can be employed much more frequently and over the entire world instead of just isolated test runs.

It’s also worth noting that there are no physics inputs to this function; there are no velocities specified for the starting point, for example. This is because The Witness is not an action game, so there are few stateful physical properties for the player. Players can’t jump, they can’t wall-run, they can’t enter bullet-time. Supporting these types of behaviors is possible with systems of the kind I’ll be discussing, but they add a layer of complexity that we don’t have to worry about on this project.

Anyhow, with DriveTowardPoint in place, it was possible for me to start on my first goal for the system: determining everywhere the player can go on the island of The Witness.

Rapidly Exploring Random Trees

Where can the player go? It seems like a simple question, but you’d be surprised how many games ship without the development team knowing the real answer. If possible, I want The Witness to be one of the few games where the team knows prior to shipping exactly where the player can and can’t go, no surprises.

This makes the problem statement, but perhaps not the problem, very simple: given a DriveTowardPoint function that faithfully determines whether a player could move in a straight line between two points, produce a coverage map showing everywhere the player could end up.

For reasons that I can’t necessarily explain, I had in my head, even before writing a line of code, that the best way to do this would be to use a Rapidly Exploring Random Tree. For those of you unfamiliar with this algorithm, it’s a very simple process whereby you record all the points you’ve visited along with a reference to the point from which you walked there. To add points to the tree, you pick a random target point anywhere in the world, select the closest point in your tree thus far, and try to get from that tree point to the target point. Wherever you end up, that’s a new sample point.

Normally, this is more of a pathfinding algorithm, so interleaved with random points, you repeatedly select some goal point as a target. This biases the exploration of the space toward the target, which is what you want when your only aim is to reach the goal. But in this case, I wanted to produce a complete map of everywhere the player could go, so I was exclusively using random samples.

After implementing this (it is thankfully a very simple algorithm to write, so it doesn’t take long), I could see that it did do a reasonable job of exploring the space (the white etchings show paths that have been explored, and the red vertical lines show places where an obstacle was hit):

However, once I actually looked at how it behaved, it was clear to me that it wasn’t really the algorithm I wanted. For example, even after many iterations, it often barely explores rooms like this one, despite densely covering the area just outside, simply because it doesn’t pick enough random targets inside it:

If I’d thought about it before I started, I would have realized that the benefit of something like a Rapidly Exploring Random Tree is that it explores high-dimensional spaces efficiently. In fact that’s the primary reason you normally use it. But in The Witness, we don’t have a high-dimensional space. We have a two-dimensional space (distributed on an intricate manifold, yes, but a two-dimensional space nonetheless).

In this low-dimensional space, the benefits of a Rapidly Exploring Random Tree are muted, and the drawback is critically bad for my stated purpose: it’s designed to most efficiently find ways to connect pairs of points in a space, not to efficiently find all reachable points in that space. If you care about the latter, then Rapidly Exploring Random Trees will actually take an excruciatingly long time to do so.

So, I quickly realized that what I should be looking for is an algorithm that efficiently covers low-dimensional spaces in their entirety.

3D Flood Filling

Once I actually thought about the choice of algorithm, it seemed obvious that what I actually wanted was something like the old 2D flood fills we used to use to fill regions of a bitmap. Given a starting point, I just wanted to fill up the entire space, exhaustively probing every possible way you could go. Unfortunately, this is a lot less straightforward in the world of The Witness than it is in a 2D bitmap, for a number of reasons.

First, there’s no clear concept of finite connectivity around a point. Everything is continuous. It’s not like pixels where you can easily enumerate 4 possible places to go from every given location, and check each one in turn.

Second, there’s no fixed size for a locus in space like there is for a pixel in a bitmap. Walkable surfaces and obstacles can be anywhere, they have no minimum or maximum feature size, and no alignment to any external grid.

Third, although walking around in The Witness locally acts as if it were on a plane, the space itself actually is a deeply interconnected and varying manifold, with walkable regions directly above other regions (stacked perhaps several times over), and special connectivity that changes based on world states (such as doors that open, elevators that go up and down, etc.)

Given these complexities, it is very easy to think you have devised a way to do a flood fill, only to find after you’ve implemented it that it gets bogged down oversampling areas, misses important paths, produces erroneous connectivity information at places where the manifold is complex, or is simply too cumbersome to use because it must be restarted to cope with changes in the world state.

No good solution to all of this immediately came to mind, so I tried some simple experimentation. Using the Rapidly Exploring Random Tree code I had written, I changed the target point selection from random selection to a very controlled selection. Each time a new point was added to the tree, I would say that points a unit step along the cardinal directions from that point where to be considered future targets, just like a basic 2D flood fill.

But of course, this would produce a useless sampling loop if I wasn’t careful. A point would branch out to a neighborhood of 8 points around it, but those 8 points would all then want to try the original point again, and this would go back and forth forever. So in addition to the controlled target selection, I also needed the simple constraint that any target which wasn’t some minimum useful distance away from an existing target point would not be considered. To my surprise, these two rather simple rules did produce a somewhat usable flood fill:

Not bad for just trying the obvious thing. But it suffers from what I’ve taken to calling “boundary echo”. You can see this effect in the following screenshot, taken while the map exploration was still in progress:

In regions with no obstacles, this algorithm proceeds nicely, sampling at a relatively even distance. But once it hits a boundary, the intersections there produce points that are “off grid”, in that they are not aligned along the sampling pattern with which the algorithm filled the adjacent open region. The reason “on grid” points don’t produce overly dense tessellation is because any new point that tries to step back onto one of the previous points finds the previous point there and declines to reconsider it. But when new points are produced on the boundary, they are completely unaligned, so there is nothing to stop them from stepping back into the already explored space. This creates a wave of offset sampling which will continue until it runs into a fortuitous line of points somewhere else that happen to be close enough to its advancing front for it to consider them coincident.

Although this may not seem like a huge problem, it is actually critical. The entire goal of an algorithm like this is to concentrate the samples in the areas where they are most likely to yield productive results. The more time you spend sampling and resampling wide open regions, the less time you spend mapping the actual edges of that region, which is the information you actually wanted. Because this is a continuous space and only an infinite number of samples can truly reveal its exact shape, your ratio of meaningful to meaningless samples is literally the measure of how effective your algorithm is at producing the walkable area.

Now, there is an easy fix for this specific problem: expand the distance at which two points are considered “close enough”. But by doing this to reduce sampling density in places you don’t care about, you lose sampling density in places you do care about, like around boundaries where you are trying to carefully probe for holes.

Localized Directional Sampling

Perhaps because I had started with a Rapidly Exploring Random Tree, my brain had been pigeonholed into thinking about proximity to the exclusion of everything else. The previous algorithms had all been about using proximity to do things, like figure out whether a new point should be considered, or which point to start from when trying to get to a new target point.

But after thinking about the problem for a while, I came to realize is that everything falls more neatly into place if you think in terms of directionality as well as proximity. While obvious in hindsight, if you’ve worked on these sorts of problems before, you know it’s easy to get trapped in one way of thinking for a while and not see the bigger picture, even if it turns out to be a simple one. That is precisely what had happened to me.

Once I shifted perspective, the correct sampling behavior was obvious. Each time I wanted to expand the exploration from a point, I would do a proximity query for a local neighborhood of existing nearby points. Instead of using the distance to those points to guide the exploration, however, I would categorize them by their direction (currently, I just use the eight cardinal directions, but I’d like to experiment with different kernels here).

In any direction where I don’t “see” a point, I walk a predetermined distance and add a point wherever I end up (whether I hit something or not). In any direction where I do see a point, I walk there and make sure I can get to it. If I can, I just add a visible edge, so it’s easy for the user to see that we’re connected. If I can’t, then I do add a new point at the collision, revealing the obstacle boundary.

This sampling method works beautifully. It allows you to very accurately control the sampling with nicely tunable parameters, keeping all the points you want while preventing unnecessary tesselation, which leads to a very fast filling of the space:

Because it searches along directions rather than just using proximity, it is immune to boundary echo, and restricts oversampling to the boundaries where you actually want it:

Furthermore, it’s completely impervious to state changes or complex manifold problems. It deals strictly with points and those points can be anywhere, and you can add new ones at any time. If you’ve already made a map of an area with a door closed, and you open the door, all you have to do is plant a single new exploration point on the other side of the door and say “continue expanding this map”, and it’ll connect itself back up and explore the entire area beyond the door properly.

You can also change any of the core parameters at any time, and everything still works. Want to sample an area more densely? Just set the default spacing lower. You can do this in the middle of a map build, and it will start sampling more densely, with no need to jettison the existing results (which may have taken some time to produce).

Rudimentary Edge Probing

Although the algorithm by default will already sample boundaries more thoroughly, because intersections produce additional points beyond the sampling pattern, it didn’t necessarily probe them as thoroughly as I wanted, because it doesn’t do anything special when it encounters obstacles. It occurred to me that since I know which points were produced from collisions and which were not, whenever two collision points found themselves connected by an edge, I could choose to invoke extra sampling to try to find more boundary points nearby.

I haven’t yet worked on this extensively, but I put in a rudimentary method to test the theory, and it shows promise. For any two collision points connected by an edge, I step to the midpoint of the edge and probe outwards perpendicular to the edge. If I don’t collide with the boundary within a very short distance, I assume that the boundary is more complex, and add a new target there so the search will continue in that area.

Even this simple scheme produces nicely dense sampling along the boundary, without any oversampling of open regions nearby. Here’s a region with several boundaries, but without edge probing:

Now here’s the same region with edge probing:

As pleased as I am with this, I’d be surprised if there weren’t significantly better algorithms for sampling the boundary, and I look forward to trying a few more methods in the future.

Early Victories

Even with little development time, and the resulting rather simplistic code, Walk Monster already produces very usable output that can find real problems in the game. Here are just a few examples of problems I’ve noticed in passing while developing the algorithm:

The slopes on the side of this extended platform should not have been walkable, but they were. This was because the player movement code has a pathologically bad way in which it deals with sloping geometry. I now know that’s in there, and I’ll be sure to fix it when the time comes to bulletproof it.

The Witness is meant to be a contemplative game, but wondering why there seemed to be a rock where there was no rock was not meant to be one of its koans. As you might guess, this was because someone left a collision volume in the game when the geometry it represented had already been removed. This is an easy thing to have happen, and it’s nice to have a tool that trivially reveals these mistakes quickly so people don’t have to worry about it.

These were supposed to be impenetrable rocks, but Walk Monster revealed them to be anything but. What’s worse, Walk Monster revealed that the path was somehow only traversible in one direction (from left to right in this top-down screenshot), which is never supposed to happen. I haven’t actually looked at this problem yet, besides verifying that I could actually do it via player motion (I could). It’ll be interesting to see what’s going on!

Open Questions

It’s encouraging to already have good results upon which we can build. Like I said, if you pick the right name for your source files, it all flows smoothly from there! But all of this work was done in the span of a few days, so it is obviously far from comprehensive and at lot of things are still completely ad hoc. If I have time to develop these systems further, there are a number of questions worth answering.

First, what kinds of post-processing on the data should I do to make it easiest to visualize? Drawing the raw network of points and edges can be very difficult for people to visually untangle, and some sort of better representation might make it possible for people to quickly understand more complex walkable areas at a glance.

Second, how can I improve the sampling patterns around boundaries to ensure that I find the most number of holes? Are there good methods for characterizing the ways in which shapes fall into a lattice, and are there good tessellation schemes that maximize the chances of intersecting and passing through these shapes?

Third, are regular sampling patterns or randomized sampling patterns better for filling spaces? I could easily modify the target point selection criteria to produce randomized patterns, but it’s not clear if this is worthwhile or, if it is, what kind of randomized patterns would be best.

Fourth, what other information would we like to get from our walkable area maps, now that we can build them? For example, it would be a straightforward extension of the existing system to do things like pathfinding or distance mapping, where a user could pick a point and ask for the shortest path between it and some other point, or to see a heat map of the distance between that point and all other points in the map. Are queries like this useful? What other queries might be?

For now, the walkable area visualizations we’re already getting from Walk Monster are more than enough to show us that the player movement code is pretty broken. I had planned to move on to creating a system for overnight testing via the user input method, but it seems clear that we already have enough problems to fix that there’s no need for that yet. So my next order of business, before improving these tools any further, will be to see what I can do to make the player movement code more robust. While I’m at it, I’d also like to see if I can get it an order of magnitude faster or two, because currently, the thing that slows down Walk Monster rather severely is the slow speed of the collision system.

With luck, I’ll report back soon in another installment on how that work is going.

The current puzzle count is 497. (This is lower than last time; last time was an overcount, as we had some temporary duplicates in the world that were being counted mistakenly.)

Right now, we save the thumbnail and the game data as separate files, but the names are mostly the same; so if you go into your game data folder in Windows you can see them living there very plainly:

(It might be interesting to do it like Spore and embed the save data in the image file, so that there's only one file per save. Maybe we'll do this, though it's not a no-brainer; right now the image file is usually bigger than the save data, so this would involve tying a relatively small amount of crucial data to a relatively large amount of cosmetic data. Maybe that doesn't matter, though.)

As I mentioned in the previous post, we are starting to get the game out to a limited number of players. It's been a long time since I've had anyone new play through the game as a whole, so I really have no idea how it plays now, and it's important to find out. However, it's not so easy, because if you sit behind someone and watch them play a game, they are going to play it differently than if you weren't there. You invalidate your own playtest data.

So, I wanted to set up the game to record some minimal playtest data, that would be easy for me to browse through later on, and that would get recorded without me being there to directly watch the play session.

One of the first approaches that programmers often consider is an input recording method. You just journal a stream of all the player's inputs, and you can play that back deterministically to reproduce the entire game session. I have never liked this method. Firstly, you need to play back using exactly the same version of the game executable that the player used, which is a pain; secondly, this kind of playback code is brittle and breaks often. An uninitialized memory read bug can render your playback data useless! (Of course, one reason you might want playback data is to track down an uninitialized memory read bug... so... there's a problem here.)

For these reasons I didn't want to use input journaling. It seemed to me that our saved games are small enough, and the thumbnails are nice enough, that together they almost solve the problem. All I need to do is save a series of snapshots / game states over time, without overwriting previous saves:

Whereas the earlier folder contained a collection of different play sessions, this one contains one play session at different moments in time. Every time the game made an autosave for this play session (overwriting the old one in the normal way), it also also copied the autosave here. The date and time that are in each filename just indicate when that game was started; the number at the end tells you which snapshot this is.

There's one immediate, very nice property of this: I don't even need to run the game in order to get a general idea of how someone's playtest went. Once they send me the files, I can just click on the first one in Windows to invoke the image viewer, and then click the forward/backward buttons in the image viewer to animate my way through the game.

But, if I have questions about what was going on, I can also dig into the save file that corresponds to any given image. I made a mode that lets me browse these saves in-game, load up any particular one into the current world state, and play from there. I invoke it by running a console command:

This loads all the saved games and thumbnails and pops them into a scrolling list at the top of the screen, in chronological order:

I just scroll to the one I want:

... then press Enter to load that game state:

It's very fast to use and it only took a few hours to program.

This whole system is also pretty cool because it's much more robust than the input-recording method. If I want to look into some detail that is super-nitpicky, I may need to use the same build of the game executable that the player used, but this will almost never be necessary. Usually I won't even need to use the in-game browser -- as I mentioned, I can just click through the thumbnails in Windows to get a basic sense of what's going on.

The fact that we only have discrete snapshots is sort of a side-bonus: because we only snapshot every minute or so, the data is not so creepy and spy-like; players can relax while they are playtesting and know that we won't later be obsessing over their every move.

Today we shipped an update, still to the same limited group. We have some initial tech issues to work out before we go wider (game packaging issues, graphics card incompatibilities, and the like).

Here's a desktop screenshot from one of the testers, with someone chiding him about what he's playing:

(The Witness window in this shot is artificially resized to make room on the desktop; by default we launch at 1280x720.)

I know that the impulse is to think "oh wow they are almost done!" Don't think that, it's not the case. We still have a gigantic amount of work to do and it'll be a while before the game is done.

Random Trivia Q&A

Q: How many footstep sound effects are in The Witness?

A: 1,119 so far. They sound really good! We will probably be in the Guinness Book of World Records as the game with the most footstep sounds...

Turns out this can be implemented easily using some of the new C++ features. I'm not a big fan of C++11; for the most part I think it adds even more complexity to an already complex language. However, I have to admit that some of the features can be useful, when used with care and moderation.

I think this is one of such cases. If you can get past the weird syntax, lambdas can be very powerful. What I find most useful is to be able to define local functions (closures) that can access the local variables of the enclosing scope.

In our case, what we want to do is to define some code that is executed at the end of the scope to cleanup objects allocated in the current scope. This is easily achieved allocating an object on the stack that invokes a lambda function that wraps our custom cleanup code:

#include <functional>

struct ScopeExit {
    ScopeExit(std::function f) : f(f) {}
    ~ScopeExit() { f(); }
    std::function f;
};

#define STRING_JOIN2(arg1, arg2) DO_STRING_JOIN2(arg1, arg2)
#define DO_STRING_JOIN2(arg1, arg2) arg1 ## arg2
#define SCOPE_EXIT(code) \
    ScopeExit STRING_JOIN2(scope_exit_, __LINE__)([=](){code;})

Then you can write code like:

int * ip = new int[16];
SCOPE_EXIT(delete [] ip);

FILE * fp = fopen("test.out", "wb");
SCOPE_EXIT(fclose(fp));

I know you can go to greater lengths to mimic the D syntax, and that Boost also provides similar functionality, but this does all we need in less than 10 lines of code.

Sadly we cannot really use this on The Witness, because I'm not sure we are yet ready to drop support for platforms and compilers that do not implement the required C++11 functionality.

Update:

As mentioned by Daniel in the comments, you don't really want to use std::function in this case. A quick look at the assembly output confirms that the generated code is horrible. It makes a lot more sense to wrap the closure explicitly and rely on type inference to allocate the wrapper object on the stack:

template <typename F>
struct ScopeExit {
    ScopeExit(F f) : f(f) {}
    ~ScopeExit() { f(); }
    F f;
};

template <typename F>
ScopeExit<F> MakeScopeExit(F f) {
    return ScopeExit<F>(f);
};

#define SCOPE_EXIT(code) \
    auto STRING_JOIN2(scope_exit_, __LINE__) = MakeScopeExit([=](){code;})

Current puzzle count: 528.

They may not look tremendously different from this off-island vantage point, but when you are down there moving around, the difference is substantial. I feel a lot better about the layout now.

One of my primary design goals, all through development, has been to keep the island spatially small, so that there isn't a lot of wandering through empty or pointless spaces. The idea is to take the best parts of an open-world game, involving choice, exploration and discovery, but compressing these into a maximally dense experience.

That has been going well, but there's an interesting phenomenon where, as we add more detail to the world, it starts to feel a lot bigger. When something is a mostly-featureless plain it is easy to feel like it's small, but once you are passing tiny rocks and plants that are set up very carefully, your brain has more scale reference. So there have been a couple of times during development where we stopped and said "Hey wait a minute, things are just too big."

Last week that happened again; we were looking at two different parts of the island on the same day, and both of them seemed much bigger than we needed. So, we decided to go back to the overall island plan and try to compress it further. We had a discussion about this on Thursday, then the architects talked it over; you see the result of those discussions in the image above. These maps are a handy guide to helping us figure out what we can cut and how we can scoot existing areas around to pack them a little better, and to better-define the areas between them.

This is a little painful, since we almost had the island back into playable shape, and this knocks it back out of playable shape for a little longer. But my hope is that we can just make a mess and then clean it up very quickly.

In other news, there had been a puzzle that I needed to design for two years -- in the sense that I knew I needed to build the puzzle in order for a certain area to be complete, and I knew what that puzzle vaguely had to be about, but didn't have any good ideas on how to build it. Last week I had a breakthrough on that and built it very quickly; it's now one of the best puzzles in the game. So things are coming along well!

The colors are all funny because we haven't written the shaders for the iPad yet, so what you are seeing is the surface normals of the objects in the scene.

We haven't written the shaders yet because they need to be OpenGL ES shaders, and the shaders we have right now are HLSL shaders all written for medium-to-high-end PCs. As the first part of the porting work, Andy made the engine work on OpenGL on the PC, and for that we converted our HLSL shaders to GLSL automatically. But the jump to ES on an embedded device is big enough that it doesn't make sense to try and use the same shaders. Instead we'll have to have some simpler shaders that we substitute.

As you can see from my previous post, I have been working on touch controls (not on the iPad, but on a touchscreen Windows PC). A very rough first cut of that is now working. It's nice to play the game with touch controls, as it feels more immediate than mouse and keyboard (and I have been a first-person shooter player forever!)

So in order to have the game fully interactive on the iPad, we just need to write the bit that translates iPad touch controls to feed them into the rest of the gameplay code, which should not take long.

Then, after that, we have to work out exactly how we are going to crunch all the graphical assets down to portable versions and do the streaming in an appropriate way... which is going to be a bit of work. Salvador built a streaming asset system a while back, and we have been using that daily on the PC, but we haven't tried to set it up on a lighter-weight portable system like the iPad yet. We'll see how it goes!

I am not sure that many people are going to use Windows for touch devices, at least in the Windows 8 timeframe. But at least some people will, so it'd be nice to have a high-quality touch interface built into the game on Windows.

With all this in mind, I started working on some Windows 8 touch hardware a couple of days ago. There is a message format that you can use in Windows 8 to get events from the touch hardware, WM_POINTER, but because I want the touch interface to work in Windows 7 as well, I wanted to use WM_TOUCH messages, which were introduced in Windows 7 and are supported in both operating systems.

It look me less than 20 minutes to get WM_POINTER working. It took many, many hours to get WM_TOUCH working, because WM_TOUCH is some kind of fully insane mess. None of the insanity seems to be documented, or discussed on the internet, so I am laying it out here in case anyone in the future stumbles into this same trap.

A WM_TOUCH message will never be returned by PeekMessage. Spy++ will tell you that WM_TOUCH is being sent to your application, but before the message is delivered, it gets stealthily converted into a mouse motion event. The type of this event will vary depending on whether you have raw input selected: if raw input, it will be WM_INPUT (0x00ff); if not, it will be WM_MOUSEMOTION (0x0200).

If your application handles these events, you may think it's reasonable to complete the handling without calling DefWindowProc. If you do this, you will never see a WM_TOUCH, because something buried down in DefWindowProc stealthily dispatches a WM_TOUCH to your application's windows when it sees a specially-marked WM_INPUT or WM_MOUSEMOVE.

Note that actually, you are supposed to always call DefWindowProc on WM_INPUT (for "cleanup" -- I don't know what this means and why they can't just clean it up when you get the next message -- but whatever); but WM_MOUSEMOVE has no such requirement, and if you believe you handled the event, it is natural *not* to call DefWindowProc. This will result in your never seeing WM_TOUCH. Strictly speaking our game had a bug, in that we were not calling DefWindowProc after WM_INPUT, but the bug was hard to find because it persisted when I tried turning off raw input, because we also were not calling DefWindowProc after WM_MOUSEMOVE (which as I mentioned is, in theory, totally fine.)

Important to note: WM_TOUCH gets dispatched to your windows without you dispatching it! You may think that you can log all messages after they come out of PeekMessage, and that you are in control of your entire message stream, but this is apparently not true. WM_TOUCH will just teleport itself into your window.

In addition to the mouse motion events, your application will get a bunch of other mouse events that are generated by the touch interface, to emulate mouse stuff for programs that don't understand touch. There is no way to tell Windows you don't want these events (unless you do Windows 8 stuff and respond to WM_POINTER; see below). According to Microsoft, the official way you are supposed to detect and ignore these messages is like this:

#define MOUSEEVENTF_FROMTOUCH 0xFF515700
if ((GetMessageExtraInfo() & MOUSEEVENTF_FROMTOUCH) 
        == MOUSEEVENTF_FROMTOUCH) { 
     // Skip message
}

I wish I were joking. It's been this way for years; Casey wrote about this in 2008.

This mask works for events like WM_LBUTTONDOWN, but it does not work for the mouse motion events e.g. WM_INPUT. If you want to take input both from touch and the mouse, you need to disambiguate the real WM_INPUTs from the fake ones. Through a process of experimentation with message flags, I landed on this:


if ((GetMessageExtraInfo() & 0x82) == 0x82) { /* ignore event */ }


It seems to work for now, but who knows if that will mysteriously break in some future version of windows.

Nicely, if your application handles WM_POINTER messages, you will not get any of this mouse emulation stuff or WM_TOUCH, so all this ugliness disappears. If it is okay to make Windows 8 your minimum specification, I recommend using WM_POINTER and staying far away from WM_TOUCH.