About the second half of the year
At the beginning of August, I left the company where I had worked for nearly eight years. The reason was that, barring any surprises, I would be going to Japan to attend a language school for a year at the end of next March. I was a little sad to leave (and to give up some income), but I resigned almost half a year early because I wanted to work on the important but not urgent things I had been accumulating over the years.
It felt as though I had only just returned home, and suddenly 2023 was about to end. I filled my schedule almost to capacity every day (8 a.m. to 9 p.m.), so this post reviews my progress in different areas during the second half of 2023.
The accumulated important but not urgent tasks
Over the years, things that seemed important to me but that I never had time to deal with kept appearing. In the end, I organized them into several main areas.
Physical health
My health-check reports got worse every year. Two years ago I had a problem with my ears; for two years in a row I was severely deficient in vitamin D; this year I had a stomach ulcer, and I have a bone spur in my cervical spine.
Strengthening my graphics-programming fundamentals
I had been working with shaders in Unity for about four years (starting in September 2019). During those four years I spent a great deal of my evenings learning: either writing Unity C# or exploring and experimenting with various shader-related features.
I started with surface shaders, then wrote custom shaders for many different effects, and later began experimenting with compute shaders and researching SRP. The process made me very interested in graphics programming. I also discovered that many ideas from writing GPU-oriented code can be brought back to the CPU side to improve gameplay logic, which makes them very practical. (I even made a plugin along the way.)
However, learning computer graphics through a game engine always means dealing with several layers of API abstraction. For example, when writing a custom shader in Unity, I can already use many engine-provided built-in variables and macros. Off-screen rendering adds a RenderTexture API to learn, and the huge number of blogs online meant I could avoid reimplementing many of the fundamentals of a modern renderer, such as shadow maps, TAA, FBOs, and skinned meshes. The result of using ready-made solutions directly is that I feel overly dependent on APIs, online forums, and blogs. I also lack the implementation experience needed to build something better than competing products on the market and to handle different edge cases.
Relying too heavily on an engine’s API creates another problem: the adjustment period becomes especially painful when changing development environments. Most game-development concepts are universal, but when I am not familiar enough with the low-level implementation, relying on an engine’s API makes me think, “The API called X in engine A is probably called … in engine B,” instead of thinking, “This API should be implemented at this point in the engine’s process.” The former is much less efficient. I have therefore always wanted the opportunity to reinvent the wheel and implement a 3D-scene renderer myself. Besides strengthening my fundamentals, I believe I will gain experience solving problems from many different angles.
Mathematics
Among the people I know who write code, I am the worst at math, without exception. I have had a mental block about mathematics since childhood, but I know one thing clearly: “If I want to take graphics seriously in the future, I cannot avoid math.” Gameplay can probably get through almost every situation with vectors and linear algebra, but graphics quickly reaches calculus when rendering equations and lighting are involved. Many GDC/SIGGRAPH papers contain no code at all. For example, the rendering paper for The Order: 1886, published by Ready at Dawn Studios, has no code that can be used as a reference for the subsurface scattering of skin’s indirect light, and it is almost impossible to find an implementation online.
There is only this pile of equations:
Being able to break down mathematical formulas and convert them into code is therefore essential. Then I also saw this troll image online:
I am the one who sucks, and it hit me hard. So learning math was something that had been sitting in the back of my mind without being done.
Playing games
It is ironic that after writing code at a game company for almost eight years, I played fewer and fewer games during that period—perhaps only one or two a year. But I love games deep down, so I decided that playing games during this gap year was important.
Japanese
After attending cram school and starting from the beginning of N4, I passed N3 and later completed the N3 course. But the certification itself is not important. What matters most is the placement test at the language school next year; I hope to be placed in a class a little above average so that the experience will be more intensive. I therefore still need to build my vocabulary as much as possible before leaving. This is actually rather urgent. XD
Progress on the important but not urgent tasks
Physical health
I started taking vitamin D, ate my meals in the park while getting some sunlight, took Bio-Three, gave up caffeine, and went to physical therapy every day in August and September for the bone spur. I also caught COVID for the first time.
But I still feel unhealthy. I suppose I will leave it to fate: life and death are predetermined, and wealth and honor are in heaven. I will just be grateful if I do not have to leave in too much pain when my time comes.
Strengthening my graphics-programming fundamentals
To understand graphics more deeply, I thought direct contact with a graphics API would be a good approach. I therefore used C++ and OpenGL to make a renderer of 💩 💩 quality. I treat this renderer as a training ground, implementing whatever feature I happen to feel like writing.
There are many graphics APIs (DX11, DX12, OpenGL, Vulkan, and Metal). OpenGL’s last major update was in 2017, but I chose it because it is easy to get started with, has many learning resources, and I currently have no need for hardware ray tracing.
Every month I record the parts I plan to implement or study the following month in a Notion to-do list.
In August and September I worked through the material on LearnOpenGL (in addition to going to the hospital for physical therapy). The site covers most of the fundamental concepts and related features needed for graphics. Finishing it did not make me familiar with the OpenGL API, but gaining a deeper, more concrete understanding of matrix and space transformations was a major benefit.
I spent all of October working with PBR: standard BRDF, IBL, Google Filament, and the implementations of different materials in UE and Unity HDRP. In the end I implemented Filament’s standard BRDF and clear coat, HDRP’s cloth material, and the corresponding IBLs. PBR theory was genuinely difficult for me—there was a lot of math, and at first I could not understand it at all. I also had to read papers about spherical harmonics, which gave me indigestion. I probably did not truly understand the spherical-harmonics material until December, when I was writing the skin shader. If I get the chance to write GI in the future, I will definitely need to review this part of PBR theory again.
Special thanks for the spherical-harmonics implementation go to TheRealMJP, a graphics-industry expert, and his Baking Lab repo. It is incredibly generous.
Starting in November, there were no longer many tutorial sites left for me to follow, so I began implementing features that interested me and reading rendering articles, such as the theory of color space, Yan Lingqi’s GAMES101—An Introduction to Modern Computer Graphics, and GAMES202—High-Quality Real-Time Rendering. I implemented basic editor features—editing and saving material assets, dragging objects with a gizmo, a scene hierarchy, skeletal animation, and so on—and also spent some time writing a CPU path tracer.
In December I added more advanced and flashy features to the renderer, including forward+ rendering with a compute shader. I also revisited The Order 1886 paper that I could not understand before. Perhaps studying math at Khan Academy had opened my mind a little; this time I understood it and used Monte Carlo integration to calculate the texture, successfully creating a skin shader. Yesterday I finally implemented HBAO before the end of the month.
Results of writing a renderer
In summary, after only a little more than four months I am still fairly unfamiliar with graphics APIs. There are many less-than-ideal API calls in the project, and I will need to keep refactoring it. But by reinventing all kinds of wheels in this small renderer, I have become much more comfortable with operations in different shader spaces. I used to do everything possible to calculate in world space first; now I feel comfortable working in tangent, clip, and view space as well.
Although both OpenGL and C++ are still unfamiliar to me, after many years of Unity development I can transfer many concepts to this renderer project. Following my simple personal rule, I store the required data in arrays or std::vector and try to finish everything in one for loop. There is little abstraction; I only think about an abstraction architecture when the code genuinely grows enough to need one. I based the order of rendering and the relationship between materials and shaders on Unity.
Another important point is that, whether I get stuck on C++ or graphics programming, I now have more patience than before and am better able to solve problems by looking through specifications and references.
I am still terrified of CMake.
Recent progress in learning math…
At first I did not know where to begin, but someone on Discord recommended Khan Academy. I found that its courses suit a math failure like me, so I chose precalculus (roughly high-school polynomials, trigonometric functions, and linear algebra) and calculus. Since November, I have spent every evening doing math on the site after finishing my code for the day. I am currently around Calculus II, although my quiz scores have been somewhat tragic. When my brain is not clear and I force myself to take a quiz, I often misread a question or make a calculation error, which leaves my final scores yellow and orange.
I think learning math has helped a lot. I still make frequent mistakes, but at least I can now understand much of the notation in papers. Rendering papers generally contain equations that someone else has already calculated or proved, so being able to understand the notation and convert it into code is enough. If you do not specifically need to read papers, then in roughly 97% of game-development situations, the mathematics from a linear-algebra course is sufficient. Reviewing trigonometry and linear algebra has also made shader writing much easier.
After reviewing trigonometry, rotation matrices suddenly clicked for me.
There are many helpful experts in the Graphics Programming Discord who are willing to answer my stupid questions. Thank you to all of them.
Progress playing games
Because I spend Monday through Friday writing code and reading during the day, I mainly get to play games on weekends. I still finished about 3.4 games: Ys VIII, Tales of Arise, Code Vein, and I am currently finishing Tales of Vesperia. I plan to play Star Ocean 6 after Vesperia, and I also impulsively bought an old PS1 game on Steam, [The Princess of Marl Kingdom 2]. It looks as though I will not have a chance to play it until next year.
I also have to complain that the protagonist of Tales of Arise loves lecturing people and is so verbose. Some of Tales of Vesperia’s systems are really… old.
Ys VIII was the best experience to play. I really want to play Ys X.
Japanese progress
This is the most urgent item, but I have prepared for it the least… I need to adjust my time allocation and spend more time memorizing vocabulary. Recently I have started watching anime with Japanese subtitles; with subtitles, I understand roughly 75% or more, and I try to read the lines using shadowing as I watch.
Because the characters’ speech is relatively limited and slow, I have watched Frieren: Beyond Journey’s End more than twice. Except for Chūta, the dialogue in Chibi Maruko-chan is also fairly normal. I mainly use these two shows to maintain my feel for the language, and I also watch Japanese news after waking up. I recently discovered that writing a Japanese diary in ChatGPT is a good way to learn vocabulary; sometimes, while writing, the AI produces words I have never seen before.
I have a Japanese conversation class every Saturday, but I feel that I speak very badly, mainly because my vocabulary is too small.
Conclusion
These past few months have been about investing in myself. I turned down both freelance work and job opportunities, and the pressure I put on myself was that the return on this investment needed to be greater than the salary I had been receiving each month. I genuinely believe this is the right decision for my career. Starting next year, I will adjust my schedule so that while beginning development on my next paid shader plugin, I can continue deepening my knowledge.