Spiritus Astrum

I ran into an unexpected issue when fixing the high-detail Level of Detail system.

The translations that I made to correctly render the terrain nodes in 32-bit space made my existing distance calculations (to the center of the nodes) useless.

It took some time to rewrite the distance calculation and get a usable distance value to use with the Level of Detail system.

I basically had to get the center of each nodes translated position (I was using it’s untranslated position originally), taking into account the radius of the world, and multiply this position by the mObjtoWorld (Object to World) matrix. This gave the correct position of that node in world space, which I could then use to get the correct distance from the player to that node.

I can now work on fixing the subdivision issues. The terrain is looking far too jagged and rough after subdivision, but I am hoping the issue is just with the parameters to the noise function.

I have finished the fixes and modifications to the terrain painter. This took a lot longer than I thought, and I may still need to make some changes to the results.

I fixed an issue with the texture coordinates not being send to the renderlist with the terrain coordinates, resulting in the lack of a texture, but the main issue was in getting the terrain textures to scale correctly based on the node height at that position.

I am not certain that I have solved this, but I will probably not know for sure until I add some more smoothness to the terrain. The textures do look much more uniform than they should, but this could be because at the moment they are mostly flat.

The next goal is to add high-detail noise to provide a smooth, natural looking terrain. This should provide a convenient test for the terrain painter as well.

P152_DEBUG
29/07/2016 , 12:11:22 AM
Torque 3D MIT – P152

I have successfully integrated the algorithm for translating the nodes, and it seems to be working very well.

The problem now is getting the terrain texture painter working with the new scaled nodes.

The textures themselves don’t show up, they display only as single colours, and the terrain painter is not applied to the correct height values. The code seems to produce a radial gradient on the XY plane, when it should be producing a height-based gradient on the Z plane (relative to the surface normal of the node in question).

I have spent some time already working on this, but it shouldn’t be a major issue to fix. Once this is done, I need to modify the subdivision algorithms to smooth out the terrain, and I can then move on to the final objectives in MS2.

I believe I have a theoretical solution to the node translation problem.

I have come up with an algorithm that essentially moves the origin of the world space node position (in 64-bit) to local space. Since all of the nodes that are being rendered are less than 10,000 meters away in 64-bit space, (They are within the 32-bit floating point precision limit) these nodes, after being transformed, end up with coordinates that are completely within the 10k limit.

This means that I can render the nodes without any coordinate or scaling factor being outside of the 10k precision limit. I have so far only tested this in gnuplot, but I am confident that the algorithm works.

The below image proves my point.

The green plot is the transformed nodes. There are between -50,000 and 50,000, instead of -10,000 and 10,000 because I increased the cull distance, but they are still renderable. The small purple blob in the center is actually what the same plot looks like when it is simply scaled from 64-bit space down to a renderable distance and scale. It is obviously much smaller than it should be in reality.

The below images show the transformed plot by itself, and the same shape in the game.

The next goal is to integrate this algorithm with the game itself, and then test it properly.

It has turned out to be a lot more difficult than I thought to implement the additional Levels of Detail for viewing at close range.

I have tried various techniques, involving duplicating the existing LOD code (which is messy) and modifying it.

The main issue that I am having is in bridging the gap between  64-bit space (which deals with real world distances) and 32 bit space which deals with coordinates within the 10,000 unit limit, as defined by the floating origin system.

I have tried implementing all LOD’s using 64-bit numbers, but with these extremely high values, it seems that the border between all LOD’s is greatly exaggerated:

P152_DEBUG
23/07/2016 , 12:54:16 AM
Torque 3D MIT – P152

This obviously does not work. It does further prove that the high-level subdivision (which I am calling “Tier 1”) is working, since adding further subdivision levels results in a terrain with vast features, ie, very high peaks, very deep valleys.

Subdividing based on the 64-bit numbers should produce this, due to their high values. However, what I want to do is nodes which are within 32-bit space, and subdivide only those nodes, in 32-bit space, to produce a smooth terrain. When the player stands on, essentially the “North Pole” of the planet, these 10K nodes form a very neat shape, which makes visualising the subdivision a lot easier:

I believe I am making progress with this. I have had to duplicate the subdivision code, which is not efficient, but my main goal now is getting the code to work, I can optimise it later. I have implemented partial “Tier 2” subdivision, but as the below images show, there are problems with nodes not rendering:

I may actually be closer to a solution than these images portray, however.

It seems clear now that I will need to add another objective to my objectives list for MS2, which is to translate the 64-bit space node positions to 32-bit. Currently, the culling and Tier 1 subdivisions are being done in 64-bit space. I am then grabbing all nodes with a 10k distance of the player, however, these nodes are being rendered with their 64-bit positions, This is causing some z-fighting issues, and will need to be fixed. I have devised an algorithm to fix this on paper, but I haven’t implemented it yet.

I may also need to combine the 64-bit and 32-bit space positions of the player together, which would be another objective for MS2. Currently, the players position in 64-bit space is used to cull any nodes outside of 10,000 units, and the players 32-bit position is used to subdivide those nodes further to produce a smooth terrain. I may be able to handle the two position separately, I am not sure yet.

I have finished updating and fixing the 3D gravity for this project, which is another of the primary objectives for MS2.

The actual Gravity code itself is quite simple, I just need to replace the existing code which applies the force of gravity only on the negative z-axis, with code that works on all axes (ie, spherical). The only real change that I had to make was to make the gravity relative to the center the planet. In MS1, the gravity was relative to the origin (in local space).

I also came across an additional problem. Even though the player is being pulled toward the correct point, there are issues with aligning them to the terrain. What I want is for the players feet (I am assuming the control object is always a humanoid player) to be flat on the terrain, no matter what the players position it. It turns out that this is very difficult, and may involve some extensive changes to the player class.

There may also be some issues with gimbal lock to deal with. I have chosen to postpone this for the time being. Currently, I am testing the project with the player standing on top of the terrain, with gravity acting on the negative Z axis, to avoid these issues, but they will need to be fixed at some point in MS2, since they are critical bugs.

I have fixed the collision issue that I was having with the terrain. This seems to have been a pre-existing bug that I never noticed before.

It took some time to track down, since some of the collision faces were working.

P152_DEBUG
20/07/2016 , 02:58:41 AM
Torque 3D MIT – P152

I used gnuplot to visualise the collision list and the render list, and they both seemed to be correct:

But when I plotted both the collision nodes and the renderable nodes on the same graph, I noticed that as the below images shows, the collision list’s nodes (in green) only contained some of the nodes that they should have.

wgnuplot
21/07/2016 , 02:47:51 AM
Gnuplot (window id : 0)

I was then able to get the collision working. For now, I am adding all nodes to the collision list, however when I implement the changes to the Level of Detail system, I will need to cull out most of the rendered nodes.

P152_DEBUG
20/07/2016 , 08:30:17 PM
Torque 3D MIT – P152

P152_DEBUG
20/07/2016 , 08:30:17 PM
Torque 3D MIT – P152

I think I am starting to see an end in sight for the second milestone of this project.

The objectives that I have left to accomplish are:

1. Fully Implement and Verify the Planet scale
2. Reimplement/fix player collision
3. Additional Levels of Detail for short view distances
4. Additional Levels of Detail and support for long view distances, until the planet is out of view
5. Collision between non-player objects and the terrain
6. Fix the 3D Gravity code (T3D applies gravity on the negative Z axis only by default)
7. Reimplement First-Person terrain rendering optimisations

I am hoping that these objectives should take no more than a few weeks to implement.

I believe I have already achieved the first objective, although some more testing may be necessary.

I am currently working on creating additional LOD’s for short view distances. My intention is basically to first subdivide the nodes from the 64-bit world space values to 32-bit values, and cull any nodes outside of the 10k range, this is currently working. I now need to further subdivide the nodes that are left to produce a smooth terrain that looks realistic and aesthetically pleasing. This may involve duplicating the existing subdivision code.

The below image is taken at a 45 degree angle between two points on the sphere, and further verifies that the terrain is at the correct scale.

I believe I have verified that the terrain is at the correct scale.

Testing this is obviously difficult, since manually traversing a realistic sized terrain is not possible. However, I placed the player at a point on the terrain, and then moved the players world position, to a point 90 degrees offset from that point, still on the terrains surface. In effect, the player had travelled from the north pole, to a point on the equator. When I did this, the terrain immediately updated to show the new subdivided nodes, in the correct position on the terrain.

I will need to do more testing on this. I am currently working on plotting more points on the surface of the terrain, and I will also need to take into account the players 32-bit position in addition to their world space position to properly update the terrain nodes. I do think I am very close with this solution, and there will hopefully not be any major problems remaining.

The next goal is to further subdivide the remaining nodes to simply improve quality and smoothness. I can then work on extending the LOD system to very long ranges, so that the player can travel from the surface to a point in space where the terrain is no longer visible. I also need to fix the player collision system, and extend it to handle non-player objects, such as AI players, environment objects, precipitation, etc.

I have made significant progress recently with regards to rendering realistic terrain. I have implemented the changes that I wrote about in my previous post, and this seems to be working. Essentially, I am taking the local space coordinates of the terrain (between -1 and +1) and scaling them to the true world-space coordinates of the planet (using 64-bit numbers). I am then performing the LOD subdivision on these 64-bit numbers, gradually iterating through the levels, and each time, I am discarding nodes that are outside a given range of the player.
The images below show this process at work:

P152_DEBUG
15/07/2016 , 06:13:05 PM
Torque 3D MIT – P152

Finally, I end up with exactly what I want, which is a small number of nodes that are less than 10,000 units from the player. This has worked very well. I still need to change the “scale” parameter in the editor to the true scale (a 64-bit number) but this doesn’t seem to cause any issues. When I scale up in the editor, the rendered nodes seem to be exactly right:

I haven’t verified the distance exactly, but they look to be approximately correct.

I still have a lot of work to do with verifying and improving this code, but I think I definitely have proof of concept.