I have completed a full retest of the third campaign, including the new story triggers, etc, that I have added.
I have run into quite a few bugs, but almost all of them were minor, and have been fixed already.
There are still some issues with triggering the story and dialog elements at the right time, but this should not be difficult to fix.
I have also made some improvements to the gui system within the project, in addition to many other minor improvements and changes.
One of the more complex issues that I have yet to solve is how to create multiple cities at the same time. Currently, this is not supported, but it needs to be in the final game.
There are some issues left to fix, and some more improvements to make, which I am working on now, but so far things are going fairly well.
There is a lot of work left, but I am definitely moving in the right direction.
I have integrated all of the dialog that I wrote last week into the game itself, as I have added triggers for this dialog so that it appears at the correct points in the story.
I still need to test it fully, which is the next step, but so far it seems to be working well.
In addition, I have fixed several bugs related to the story and dialog system, and the guis, etc.
The dialog system has some basic random elements built in. Character names, etc, will be changed between playthroughs. In the future, I intend to extend this to add more replayability, and more procedural content.
The dialog for each NPC will also change during the story, depending on the game state, and the players actions, etc.
I have also started extensive testing of all of the changes that I have made recently. This will take some time, but progress so far has been good.
I have added several more mission objectives and events for the third campaign.
I have also written over 8 thousand words of dialog and mission briefings for the campaign, and this is just a first draft.
I am now working on integrating this dialog into the code.
I have added a menu to display the mission briefings, and I have made improvements to the dialog system to display dialog more easily during the missions.
There is still quite a bit of work to do here, but I am making good progress.
The main goals left are the artwork, and lots of testing.
I have spent this week making extensive improvements to the missions for the third campaign. The goal now is to add more content to them in the form of new mechanics and objectives, etc. Once this is done, I can add the artwork and story elements.
I have added new enemy types, as well as new placeholder art and objectives, etc.
I have also made several improvements to the combat system, and fixed several bugs, etc.
There were some issues that cropped up when creating large numbers of enemies at the same time, as well as some issues with placement and targeting within interiors, but these issues seem mostly solved now.
I have added several more missile types to the game, with different stats, etc, and I have improved the projectile-based weapons as well. There is still a performance issue when the guns fire a large number of projectiles at once, even with object pooling. I will need to look at this later.
I have also added the ability for the player to add multiple guns to their fighter’s hardpoints, in addition to missiles. This does cause issues with performance, as I said, but it also allows for more variety in gameplay.
Finally, I have added an upgrade system where certain weapons and items are unlocked after specific missions.
The next goal is to add more mission objectives and content to the missions themselves.
I have created a short demo video showing the space sim mechanics here. This is early footage, the artwork is not final.
I have spent about a day or so working on writing code with DeepSeek-V3-0324 (I am running the model locally. For information on how I setup this model, check out my post HERE).
This is actually my first time experimenting with AI generated code, and the results were, in a word, spectacular.
With very minimal prompting, the AI was able to output quite complex and complete code, with very few issues or bugs.
I used the AI to generate a C# Forms application to display the Mandlebrot set, display a rotating 3D cube on the screen (A Tesseract), display a demo of Conway’s “Game Of Life” and it’s “Brians Brain” modification, as well as write a Base64-to-image converter.
All of these programs worked very well, and where issues occurred, I was able to quickly resolve them using plain english.
This is an example of the output for the first program, to display the Mandlebrot set. The query was:
“I would like you to write me a program in C# that displays the mandlebrot fractal set. The program should be as concise as possible, and should ideally use no external libraries, however it should display the fractal in colour on a gui.”
Output:
“I understand the requirements – a concise Mandelbrot fractal generator in C# using only built-in libraries while still producing a colored GUI output. This is an interesting challenge given the constraints.”
“Here’s a solution using Windows Forms (System.Drawing) which comes bundled with .NET, requiring no external dependencies:”
using System;
using System.Drawing;
using System.Windows.Forms;
class Mandelbrot : Form
{
const int MaxIterations = 1000;
const double Zoom = 300;
public Mandelbrot()
{
Text = "Mandelbrot Set";
ClientSize = new Size(800, 600);
DoubleBuffered = true;
Paint += (sender, e) =>
{
for (int y = 0; y < Height; y++)
{
for (int x = 0; x < Width; x++)
{
double zx = 0, zy = 0;
double cx = (x - Width/2) / Zoom;
double cy = (y - Height/2) / Zoom;
int iter = MaxIterations;
while (zx*zx + zy*zy < 4 && iter > 0)
{
double tmp = zx*zx - zy*zy + cx;
zy = 2*zx*zy + cy;
zx = tmp;
iter--;
}
Color c = iter == 0 ? Color.Black : Color.FromArgb(
iter % 8 * 32,
iter % 16 * 16,
iter % 32 * 8);
e.Graphics.FillRectangle(new SolidBrush(c), x, y, 1, 1);
}
}
};
}
static void Main()
{
Application.Run(new Mandelbrot());
}
}
“This implementation:
Uses Windows Forms for the GUI
Implements the Mandelbrot algorithm directly
Colors points based on iteration count
Is completely self-contained (no NuGet packages needed)
Compiles to ~100KB executable
The zoom factor can be adjusted for different views”
“Would you like me to explain any particular part in more detail? Or perhaps suggest optimizations or modifications?”
I then asked the AI for detailed instructions for how to run the code:
“Excellent, thank you. How can I run this code, do I just create a new blank project in visual studio?”
And the response that I get was equally intelligent:
“Yes! You have the right idea. Here’s a precise step-by-step guide:”
Open Visual Studio
Create new project → “Windows Forms App (.NET Framework)” (Important: not .NET Core for this example)
Replace ALL code in Form1.cs with the Mandelbrot code I provided
Click the Start/Run button (or press F5)
“Key technical notes:
The code uses .NET Framework’s System.Drawing which isn’t available in .NET Core by default
The project template must be Windows Forms, not Console or WPF
No NuGet packages need to be installed
Compilation happens automatically when you run”
“Would you like me to modify it for .NET Core if that’s your preference? Though we’d need to add a NuGet package for System.Drawing in that case. Or perhaps you’d enjoy seeing a real-time zoomable version?”
I needed to make some changes to the code in order to get it to run properly, and when I posted my finished, working, version of the code back to the AI, it was even able to suggest improvements:
“I see what you’ve done! You’ve:
Created a proper Windows Forms project structure
Wrapped my Mandelbrot class inside the Form1 container
Commented out InitializeComponent() since we don’t need designer-generated code
Initiated the Mandelbrot form directly from Form1’s constructor”
“Your changes are quite logical. Though may I suggest two slight optimizations to your version?”
// Instead of Application.Run inside Form1's constructor,
// modify Program.cs like this:
static class Program
{
static void Main()
{
Application.Run(new Mandelbrot()); // Run Mandelbrot directly
}
}
// And make Form1 empty or remove it entirely
“This would:
Eliminate the unnecessary Form1 middleman
Follow Windows Forms conventions more closely
Prevent potential double-form issues
Make the project structure cleaner”
“By the way, I notice you kept all the default using directives – would you like me to strip those down to just what we actually need? We could reduce it to just System, System.Drawing and System.Windows.Forms for better clarity. Three simple namespaces would suffice!”
This is very useful feedback, and the AI even successfully caught the unneeded “Using” directives, I have a having it leaving those in while debugging, having the AI suggest removing them is exactly what a human programmer would do!
The AI also suggested changes to the programs, such as the “Brians Brain” variant of Conway’s Game of Life, which I hadn’t even heard of! It then either modified the program with those changes in mind.
In one case, the changes became so numerous that the AI offered to completely rewrite the program to avoid confusion, which it duly did!
I am confident that I could have written the above programs myself, just relying on internet research and my own knowledge to help me, however, it would certainly have taken far longer. Working with the AI was a substantial time saver, it really was like having another person there to assist me with questions, even complex techical questions.
I am currently experimenting with using the AI to write a much more complex program, (I want to take the system audio and display an aesthetically pleasing visualisation of it on the screen) and this is going well, but it is a lot slower. I feel that with the more complex program the advantages of using AI to code are not as strong, I find that I can probably fix mistakes faster than waiting on the AI.
I think that this is a very important point to make regarding the use of generative AI in the workplace, and in society. Simple tasks (such as simple programs) can be easily offloaded to the AI, however when it comes to more complex tasks, much more human interaction is needed.
So, in the future, as AI becomes normalised in society, we could simply use AI for the menial tasks, freeing up valuable Human Resources for creative and complex tasks. The idea would be that instead of AI “replacing” humans, it is instead enabling humans to do what they do best: Create, Understand, and Produce, while the AI simple handles the simple boring time-wasting tasks.
I have uploaded some of the examples of the AI-Generated scripts to my github. To access the scripts, click on the links below. The Scripts are written in C# (.net).
I have spent this week working on improving the combat system mechanics.
I have fixed some bugs and made some improvements to the targeting system for guns and missiles (There was a minor issue where multiple missiles could not be fired at the same target).
In addition, I made some improvements and ran some tests on the checkpoint save system for the third campaign.
The vast majority of the bugs and issues that I discovered during the last test run are now done.
The main goal for this week though was adding in the loadout selection system for the fighters.
It is now possible to choose which weapons the player wishes to equip on their fighter, and this loadout will be saved persistently.
I have also had to make changes to the HUD and combat system to allow the player to view and select their loadout, and fire missiles, etc, from specific hardpoints based on their selection.
I still have some more missiles types and other weapons and equipment to add to the game, but the loadout system works very well.
I have completed another full retest of the third campaign in the game.
There were quite a few bugs and issues identified, but thankfully, nothing serious. There were no issues with the core mechanics, pathfinding, etc.
Most of the bugs that were found have already been fixed.
There were some issues found with the combat system, I will need to spend more time on this in the coming weeks.
The next goal is to fix some of the bugs that have not been fixed yet, and work on improvements and tweaks to the third campaign. I should also be able to work on more artwork and more content for the campaign, before doing another full retest.
Overall, the third campaign is going very well, most of the major issues should be fixed by now.
After hearing good things about this model, I tried, and succeeded, in getting it to run locally.
For those unaware, DeepSeek V3-0324 isn LLM developed by the Chinese Based Company DeepSeek.
The V3 variant is a chat model, as opposed to, for example, R1, which is intended more for reasoning.
The GGUF Q8 version of this model is around 700 GB’s in size, much larger than the 120B (Q8) models that I have been running up to now. However, DeepSeek is a MOE, a Mixture of Experts model, and only has 37 Billion active parameters, meaning that it can be run on systems with less VRAM.
Crudely speaking, MOE’s trade space for compute. IE, MOE’s tend to be much larger than “Dense” models, however, if you can fit them into RAM, they run much faster.
My AI Rig currently has an A6000 Ampere and two 3090s, for a total of 96 GB’s of VRAM, and it has 256 GB’s of DDR4 RAM in 8 Channel mode, and a Threadripper 3975WX CPU.
I am running Unsloths GGUF models, available from HERE.
This gives me a total of 352 GB’s of RAM, which is just enough to run the Q3_K_XL quant of this model.
Q3 is not particularly high, but for a very large model like this, it should be usable.
What follows is my experiences with setting up and evaluating the model.
Firstly, I ensured that both KoboldCPP and SillyTavern were both updated to their latest release (1.93 for Kobold, and 1.13 for SillyTavern). The latest versions of KoboldCPP, in particular, are needed to load MOE models.
The model itself is loaded in the same way as any GGUF model.
The Unsloth GGUF that I downloaded has 62 layers, I was able to offload 17 of them to the GPU’s VRAM, using a tensor split of 0.9,2.0,1.0.
Please not that in later versions of KoboldCPP, the GPU order has changed to the PCIE Bus ID, which, for me, was different to the previous load order (Please see the issue I opened HERE on Github). This meant that I had to change the order of the tensor split.
Once I did this, I noticed that the model was loading successfully, and KoboldCPP was starting, but it was failing with a “CUDA error: out of memory” whenever I would start inference. This was caused, I believe, to the context being too long.
I don’t know if MOE models in general require more RAM for context, or just DeepSeek.
After several days of tinkering, I managed to come up with some settings that worked well for my system.
The main issue for me was the “BLAS Batch Size” under the hardward tab in KoboldCPP. This was set to 512, I changed it to 64.
The BLAS Batch Size has a huge impact on the speed of prompt processing, and this makes and even bigger difference for MOE models due to how they work. A Batch size of 64 makes for VERY slow prompt processing, but once the prompt processing is done, the generation time is very fast.
I wanted to have at least 32k of context, since this is the minimum that I am used to dealing with. I believe deepseek supports up to 128k context, which is huge, but I haven’t verified this.
I also had to set “Quantize KV Cache” to “8-Bit” (Default was FP16) to prevent OOM errors.
I also set the Context to 32k in SillyTavern as well, to match KoboldCpp.
My settings now look like this:
KoboldCPP v1.93 Settings for DeepSeek
GPU Layers
17/62
Context Size
32768
Use ContextShift
On
Use FlashAttention
On
Tensor Split
0.9,2.0,1.0
BLAS Batch Size
64
Quantize KV Cache
8-Bit
As a result of all of this, I have managed to get DeepSeek V3 0324 Q3_K_XL running on my system with 32k context.
Almost all of my VRAM and System RAM is used up with these settings!
Prompt processing is very slow with the 64 bit batch size, and I am currently looking into reducing the layers being offloaded to the GPU to free up memory, so that I can increase the Batch size to at least 128.
However, the token generation is much faster!
With context set to 2k I was getting 8.51 tokens per second!
2k is obviously too low to be useful, and as I increased the context size, and the context filled up, token generation rates droped to about 4-5 tokens per second, and then, at 32k with some of the context filled, I am getting around 1.5 tokens per second at least, and this is including the slow prompt processing times.
This is not ideal, but it is still faster than what I would get with a dense model.
So, how good is DeepSeek compared to the 120b Dense models that I have been using?
Subjectively, the quality of the writing seems far superior. The Model seems to follow the prompt better, and characters in creative writing exercises seem to have a lot more personality, and feel more unique than other models.
Even though prompt processing is slower with my settings, token generation is much faster,.
I am limited by my available system RAM at the moment (I should have used a Q2 quant, not a Q3 one), and I am considering a RAM upgrade to 512 GB’s.
This presents an interesting question. 512 GB’s of DDR4 RAM (At 3200 MHZ) would cost about the same as anothe Power supply and another 3090 for my current rig, so, which would be better?
For running dense models, the extra 3090 would be much more useful. With 120GB’s of VRAM I could run 120b models at Q8 almost entirely in VRAM (Maybe with a little spilling over into system RAM at high context sizes) at very high speeds. However, for running MOE models, I would be far better with the 512 GB’s of system RAM (I could probably run a Q5 of DeepSeek, with much higher context and without quantising the KV cache. I could also use the GPU’s just for prompt processing, further increasing speed).
So, the question is: Are MOE models generally better than Dense models?
I don’t think there is a hard and fast answer to this.
There are some that argue that MOE models are inferior to dense models in terms of quality, and that they are only popular due to the fact that they can be run on cheaper hardware (RAM and compute is a lot cheaper than VRAM), and that if you can run a large, dense, model, that would be the better option.
However, there are others that might argue that dense models will, or have, hit a wall, and that Mixture of Experts models are a necessary step to continue to improve the quality of models.
I have come across a formula on reddit that seems to provide a rough estimate of the performance of a MOE model compared to a dense model. It is:
sqrt(active parameters * total parameters)
This would seem to indicate that DeepSeek with its 700 Billion total parameters, and 37 Billion active, would be generally equivalent to a 160B dense model. Based on the few days that I have spent working with it, this would seem accurate.
The model feels powerful, and, subjectively, it feels superior to 120b models, but not six times better.
Time will tell where MOE type models become the future or not, but even if they dominate the high end LLM space, Dense models (Including distills of MOE’s) will likely still have a place in the sub-70b space for a long time to come.