“It’s not optimized yet.”

This post is my day 14 contribution to the 2015 Blaugust Initiative.


Every time I come across this phrase, my jaw starts to clench up, my knuckles turn white, and I start to hear my heart beating in my ears. I’m normally a very soft-spoken hobbit, but when someone uses “It’s not optimized yet” as an excuse for poor game performance after months or years of testing, I begin to feel a little swear-y.

It’s probably worth noting that in 20 years of testing games, I have not yet come across a single reputable game developer using this as an excuse. It’s almost always uttered by fans of a game in defense of questions or criticism from other testers. I picture them fluttering their fingers on the other side of the keyboard, like it’s some kind of Jedi mind trick. “This is not the finalized performance you’re looking for.”

Clearly, it's just not optimized yet. CC BY-SA Patrick Mannion on Flickr

Clearly, this game just isn’t optimized yet. CC BY-SA Patrick Mannion on Flickr

It’s true that most game studios push their developers too hard, and this can lead to sloppy practices and lazy shortcuts. The imperative from management is sometimes just “write code that works,” rather than “write good code that works well.” In studios that value quality over deadlines, the developers and QA team will identify and either correct or remove any changes with a noticeable negative impact on performance before customers ever see it.

It’s also true that some studios will occasionally ask their developers to do an “optimization pass” to look for ways to improve a game’s performance. This should be reserved for taking games that already perform acceptably well, and making them perform even better. An optimization pass creates a cushion for future changes that could drag the game’s performance back down to merely acceptable.

If you’re using hardware and software within the targeted specifications given by a game’s developer, and that game doesn’t perform acceptably well during months or years of testing, it will almost never perform acceptably well for you upon release. The only exceptions I’ve seen are when a developer already has clear plans to take advantage of some additional technology prior to launch, such as multi-threaded rendering or previously unutilized GPU functions.

The only remaining alternatives are to eliminate game features entirely or reduce graphical quality across-the-board to reach acceptable performance goals. But that isn’t optimization; it’s amputation.

So for you passionate players out there, please stop trotting out this ridiculous excuse for your favorite game’s poor performance during testing. It’s almost never true. And developers, unless you honestly have an ace up your sleeve, please set the record straight when you see someone say, “It’s not optimized yet.” Your silence only perpetuates the fantasy.

Am I completely wrong here? Has some magic optimization bullet saved your game from subpar performance the day before release? Set me straight in the comments below!

Advertisements

6 thoughts on ““It’s not optimized yet.”

  1. I didn’t go into factors that would understandably decrease performance in a testing environment, such as debug code, extremely verbose logging, or real-time telemetry. Ideally, these are things public testers should opt into when they encounter specific issues. But even when they’re mandatory, customers should always be made aware of them, so they can take these factors into account when evaluating a build’s performance.

    Like

  2. Trap sprung, here I am!

    First, a caveat: I agree mostly with your premise, because of this caveat you mentioned: “after months or years of testing”. If the game plays like shit and hasn’t been fixed in months/years, yeah, you’ve probably got a lost cause. You’re probably well aware of this, but the longer code sits, the harder it becomes to change it. Macro-optimizations are easiest to implement either at the design stage, or right after the feature was implemented. Once the code has sat, other systems rely on the broken behaviour and the folks who wrote it originally totally forget how it worked. Changing it becomes a risky proposition, possibly with a large bug tail.

    Also, as you mention, logging definitely trashes perf, especially if every statement is grabbing a full callstack coughUnitycough. And debug code.

    That all being said, it’s very, very, very easy to write bad code, or for devs (designers and programmers both) to abuse features in way that tanks perf significantly.

    An example from my own experiences, we’re using the physics sub-system to track when players/camera/enemies/attacks/etc. hit things. We have trigger colliders everywhere. We’re talking in the realm of 2000+ in a single level. When they were all colliding with each other, it was generating 8000+ collisions a physics frame, which tanked perf. I spent 2 days designing and implementing a way for us to divvy those colliders up in a way to reduce the layers that were colliding with each other such that two trigger colliders could be overlapping say, and not actually ever interact because they didn’t need to; i.e.: a camera zone collider didn’t care about an actor’s combat collider. This 2 days increased performance of the system 1600% to 3200% depending on the level.

    This was about 6 months into development, and the problem had been identified 3 months before hand, we just never got around to fixing it until it became crippling (the perils of being understaffed for the product you’re building). So there are exceptions to the rule. I also improved our save system performance by 300,000% (no, not a typo) 6 months after the original implementation, though that required throwing out the old implementation entirely and rewriting a new one. Also “easy” because save/load is mostly a leaf feature. Not many systems sitting on top of those.

    So really it depends on where/how the perf degredation is coming from. Is it an easily identifiable bottleneck that can either be rewritten or redesigned? Salvageable. Is it death by 1000 papercuts? You’re honestly probably hosed and need to take drastic measures. “It’s not optimized yet,” if a developer says this, and then says yeah, it’s a debug build because they couldn’t get the ship build running in time for demo land, probably the truth.

    If a demo build drags a bit, as a developer I’d not be terribly concerned. If it’s running at 2 FPS, then yeah, you’ve got major problems and WTH are you demoing?

    Liked by 1 person

    • Yay, a comment by a real game developer! It’s reassuring to know that breakthroughs can happen when you have good people working on it. I just wish more studios would be more honest in managing their customers’ expectations, either by admitting existing limitations and sharing plans to overcome them, or by publishing more realistic system requirements for their games. For example, include the average sustained FPS that can be achieved with both the minimum and recommended requirements.

      Like

      • It’s funny, sometimes I still don’t feel like a real game developer…

        Admitting existing limitations would be a good idea, especially for demos. Sharing plans to overcome them I think is overkill, especially since for performance, it’s easy to identify bottlenecks, but fixing them is much harder (which is the opposite of most other bug types, where finding the root cause is hard, but fixing them is often easy).

        Average sustained FPS is a hard sell. Too many things not related to your hardware and the game can affect those. AV running? Virus chewing your CPU? Hardware is slightly bad? Also, with the sheer number of hardware configurations, you’d never be able to nail down a number. Intel processors do different things better/worse than AMD processors, for example.

        Like

      • How do studios actually set system requirements? Do they have targeted performance goals and choose combinations of hardware and software that meet those goals during testing? I know at least one studio that does hardware surveys and just sets minimum requirements at the 20th percentile and recommended requirements at the 60th percentile.

        Like

      • Good question. If you find out, let me know 😛 I get told what hardware we want it to run on, I say, “Sure/Maybe/Nope” and if it’s in the Maybe we test for it. If we can get settings/optimizations to make it run on the maybe machines, then hooray. Otherwise, we drop them.

        Not a precise science, but we’re also a tiny studio. I have no idea how other studios do it.

        Like

Comments are closed.