Upon further inspection I realized that the pseudo-inverse stuff is irrelevant because the matrix diagonal will always be nonzero. It should always be invertible, even with the compact support factor cooked in the matrix (which is crucial).
The upshot is that the outlined solution should be easier to verify.

Like you said, zeroing a row of the matrix makes it rank-deficient. However, a valid ‘variogram matrix’ (and it’s inverse) is symmetric, so if a row is zero, the corresponding column must also be zero, in which case it is still pseudo-invertible (as if lower rank).

With that in mind, consider that even if a control point’s immediate influence doesn’t reach a given terrain block, it will reach other control points in the block’s vicinity, and so the matrix will have nonzero elements somewhere along the row and column of that seemingly-cullable distant control point.

If all that is correct, the solution would be to include in the matrix for a terrain block all control points within r1 of any control point within r1 of the block (i.e. the neighborhood of the neighborhood).

(First make sure that the compact support factor is included when calculating the matrix, not just applied to the final weight vector. I assume it was, but that wasn’t clear from the article.)

A quick test, assuming you have the means to compute pseudo-inverse instead of strict inverse, would be to expand the cull radius to ‘2*r1’ instead of ‘r1’ (some null rows/columns might result).

Sorry I’m very late to the convo, but stumbled in while investigating similar things and thought that might be worth sharing.

The problem you reached was a result of trying to multiply by zero….what if instead of clamping the weights between [0, 1], have them asymptotically approach zero? That’s the first thing that popped into my head.

I see others have mentioned natural neighbour interpolation below. You can see a picture of the Prussian helmet effect on page 109 of http://technomagi.com/josh/math/phd.pdf .

Interpolating surfaces suffers from the “Prussian helmet effect”. See:

Robin Sibson. A brief description of natural neighbour interpolation. In Vic Barnett, editor,
Interpreting Multivariate Data, chapter 2, pages 21–36. John Wiley & Sons, 1981.

which also gives a good interpolation scheme for scattered data (based on Voronoi diagrams), assuming you have some way of estimating the tangent plane at your data samples. And once you have the tangent planes, the interpolant is very local.

I’m developing a system which will use the Kriging method for displaying data. I would be interested in the code you referred to above. I’ve not been able to find any working implementations other than a mathcad library, so if you know of any other existing code (the universities code that you reference above, for example), I would be interested in looking at that, too.

I used both kriging and RBF in my application, but still have problems with local update of the surface.

I have tried to build some polygone around the updating region. Then I built so-called smoothin zones where I lineary smoothed base surface and new one. But this approach gives some artifacts.

You can see small pdf document with description of my method: http://ubuntuone.com/p/WVc/

However, the idea still does have applicability here; if one wanted to use some other kind of tool to sculpt a local area, it would then be interesting to apply a find-the-kriging-weights system to that in order to generalize it.

Then you could pick up random areas of terrain and move them/rotate them, apply the changes back to the terrain, and be done with the control surfaces. Looking up “inverse kriging” brings at least one promising hit…

Could you give some more detail re what the artifacts you see with compact support are? I’m curious to see if my applet has the same issues or not. (Unfortunately the springerlink papers seem to be behind a paywall …)

In general, though, anything that tries to re-use prior solutions, etc, etc, I don’t have patience for at the current time — it is way too easy to rathole on tech (as discussed in the comments, the shadow map memory savings thing is sort of that, though it might help on consoles). To have any prayer of shipping a game any time soon, we just have to do the simplest things possible wherever possible.

Of course sometimes it turns out that you need to do a lot of work in order to establish basic functionality. If that turns out to be true, then in this case I probably would pursue it, because this way of modeling terrain really is the right thing for this game. So, we’ll see…

Also, have you considered solving a discretized version of the problem? Often when I’ve seen people implement thin plate splines for shape modeling (which are essentially the same problem as this I think?), instead of solving for the function weights directly they turn it into a finite difference or finite element problem and solve for the vertex positions that way. It ends up giving a giant sparse matrix (one equation per free vertex), instead of a smaller dense matrix, but since sparse solvers scale much better it can turn out to be faster depending on the problem. Also, as long as the constraints don’t change too rapidly during editing, you can get a speed boost by reusing previous solutions (or previous factorizations, for sparse direct solvers).

In general, though, I care more about the game than about what platforms it is coming out on. So if we needed to do something for the game to reach its full potential, and that thing doesn’t work on some specific platform, then the answer is probably just not to release on that platform (or else to release an inferior version on that platform).

I was working out a more-formalized way to get compact support in kriging. But even when I did it the supposedly right way, I still saw some artifacts. Researching why led me to the following complaint by Philip and Watson:

http://www.springerlink.com/content/kh71041n26070768/

(Watson is the very same David Watson mentioned in these comments by James McNeill). This referred to another paper that shows discontinuities that happen with kriging:

http://www.springerlink.com/content/r73v34635g6m1718/

Which looks exactly like the artifacts that I get when I don’t try to mitigate them, so I know that my kriging code is working properly and that we are on the same page.

This complaint and other follow-up paper were encouraging the use of Natural Neighbor Interpolation as a way of solving these problems, so I decided to try it.

With some searching I found Pavel Sakov’s nn library for Natural Neighbor Interpolation: http://code.google.com/p/nn-c/

I got this integrated into the game and working fairly quickly. The results are better than I postulated earlier in this thread (indeed, triangles are not quite faceted, I didn’t understand the algorithm so well when posting that comment). However, while not quite faceted, the shapes generated by natural neighbor interpolation are not so natural, at least for a sparse set of sample points. The results from kriging look a lot better with many fewer sample points. So now I am back to kriging and working on ways of minimizing the artifacts and speeding it up. Seeking some outside help from experienced geostaticians in this area.

On top of this, not very many people used the Braid editor. This may be largely due to the fact that it is not explicitly called out in-game, with a menu option or something. But, in order to promote an editor as a feature at that level it has to have way, way more work put into it than made sense at that time. I would expect the same thing to be true for The Witness.

My current thinking on this is, if your game is designed around user-made content, then it totally makes sense to make the editor great and expose it in a high-profile way. Otherwise, it doesn’t make sense, because the number of people who will use such an editor is not very high, and pragmatically speaking, you can add more to the world / create more interesting times for more people just by working on the next game.

To speed it up, I think for RBFs with global support people have used the Fast Multipole Method to do faster solvers. Alternatively, for compact support interpolating implicits, perhaps check out the “Multi-level Partition of Unity Implicits” paper?

How many control points do you use for your whole world?

James: Interesting, but I worry about the idea for this kind of application. If you are inserting your samples into a Voronoi diagram, then you are always interpolating between only very-close neighbors. In cases where you are interpolating on a triangle or near-triangle (which are pretty common in Voronoi diagrams), you are going to just be linearly interpolating between 3 points, i.e. making a flat plane. So the result would exhibit hard facets very often. I think the estimation-of-normals you are talking about would be key in eliminating that problem… Certainly though, this method would be a lot faster than what I am doing, and if the smoothing makes it usable, that would be cool.

Marc: As I mentioned, I want to interpolate points. I don’t want to treat each point as an individual summed Gaussian or whatever that moves around, because that would be unmanageable, and it’s very difficult in that kind of scheme to get smooth surfaces of the shape you want without endlessly tweaking the widths and heights of neighbor points… and then if you want to make a small change, say hello to more endless tweaking. No good.

http://en.wikipedia.org/wiki/Natural_neighbor

The basic idea is to build a Voronoi diagram of the control points. Then, for each sample point on the height field grid, you pretend you’re inserting it into the Voronoi diagram. The sample point is a weighted sum of the control points from which it would have stolen area; the weight for each control point is how much area it’s contributed to the new Voronoi region.

This algorithm has local support built in so updates could be made fairly efficient. The surface is smooth everywhere except at the control points. It works well with anisotropic sample data; for instance, if you have sampled the terrain in long strips of data points, i.e. Mars orbiters:

http://pirlwww.lpl.arizona.edu/~abramovo/MOLA_interpolation/interpolation.html

I initially encountered natural neighbor interpolation years ago via Dave Watson, a professor at an Australian university. Unfortunately he seems to have dropped off of the web. I remember he had a modification to this algorithm where you could estimate (or define) normals at the control points and add those into the interpolation to get rid of slope discontinuities there.

I still haven’t dug into the Kriging algorithm and obviously have no idea how your terrain system works, But I do have one comment/suggestion regarding ‘compact support’ you mentioned:

Did you try interpolating surfaces instead of weights?
What I mean is, you could generate one Kriging matrix for each of the terrain blocks containing the control points from that block and extending into adjacent blocks (you can make the overlap as big as you like by increasing the radius of interest).

When trying to evaluate the height of a sample you need to sample each ‘Kriging surface’ that covers your sample and interpolate between the different surface results. Just need to figure out the weight for each surface which shouldn’t be difficult. If anything that should assure continuity.

If you have an overlap of 1/2 a terrain block (choose r = grid_block_size * 1.5) you could be done with just sampling 4 surfaces. On the down side, you might loose some local support away from the grid block centers – but with an overlap of a full terrain block you still get just 9 samples which should still be pretty fast and practically quite accurate.

Anyway thanks for all that info, cool stuff!

So when you are solving for one specific sample point of the terrain, you only think about that sample point and the control points; you don’t think about any other sample points. (This is a little different from the geostatistics way of thinking about it, where the control points are some of the sample points.)

Seems like something like compressed sensing / basis pursuit could work pretty well too. http://en.wikipedia.org/wiki/Compressed_sensing

Thanks!