ROBOT-SB dev blog – “pixel perfect” part 3

Welcome to the third installment of the “pixel perfect” series. This is the one where I’m supposed to quit stalling and actually talk about the subject rather than more boring preliminary background material (BPBM).

What?! You don’t like BPBM?!?! Yeh, ok, I’m with ya. Though, by the way, there’s plenty of BPBM in part 1 and part 2 if you’re into that sort of thing, in case you missed it.

Feel free to just download the source for example 1, example 2 and example 3 if you’re the type who does better studying actual code and would rather skip all the reading.

Now, on to the TLDR;…

First, a quick reminder of the goal: to create a “low-resolution retro blocky pixel” configuration for Corona SDK that displays properly and without aliasing artifacts on a wide variety of target devices, resolutions and aspect ratios.

Specifically, for my purposes, the “content resolution” should be 160 units wide, by whatever number of units high needed to match the device aspect ratio:

Note that there’s nothing “magical” about 160 specifically, other than it’s convenient for this project. It’s similar to Game Boy resolution, and a nice-enough divisor of common mobile device resolutions, that it was worth adopting, at least as a baseline. So that’s the dimension I’ll use for the rest of this discussion.

(Besides, if I tried to keep this purely theoretical and avoid selecting a specific resolution, then I’d probably never get anywhere with regard to presenting an actual working implementation. Just endless BPBM. So, adapt this discussion to other specific dimensions if/as you see fit.)

In previous installments I mentioned that something, somewhere will have to be “flexible” in order to accomplish this goal across the wide variety of devices. So, in this installment I’ll present two approaches: a simpler one, that “mostly” accomplishes the goal, with perhaps-acceptable compromises; and a more complex one, that does accomplish the goal, though with additional considerations that will have to be dealt with in program code.

Throughout this discussion I’ll be assuming portrait orientation just so I can adopt a single set of terms. But all of these concepts will apply similarly to landscape orientation.

The Simple Approach

If you carefully apply the foundations (ie, the BPBM) covered in the first two installments, then this approach can work pretty good under some conditions, and is so simple as to almost not warrant any discussion at all! But, it is not “pixel perfect”, except on a limited range of devices. What it is, actually, is “pixel-acceptable-but-does-preserve-fixed-content-width”. However, there are aspects of it that will apply to the more complex solution, so it’s worth covering first and explaining a bit before presenting something more complex.

“Wait just a gosh-darned minute,.. so is this so-called ‘simple approach’ just another sneaky way of wasting on our time on more BPBM?!?!”

Good for you, you’re starting to catch on.

Here’s the config.lua file:

And main.lua (for the on-screen results presented a bit below):

Or, grab the entire source code for this example.

The only thing even a little bit “tricky” in there is calculating the height based on the device’s aspect ratio.

(I’ve have been doing aspect ratio height calcs since my first encounter with Corona SDK, even when using more-typical content dimensions, just to prevent screenOriginY and contentCenterY issues, so it has long since lost it’s “magic” for me — it would be helpful if you grok it well-enough that it ceases to seem “magical” to you too, at least by the time we reach the complex approach, just so that it can be taken for granted and ignored.)

But another thing worth noting is that “left”/”top” alignment is used rather than “center”/”center”. This is done to assure that the alignment of the content coordinate grid at least begins (at 0,0) in alignment with the hardware pixels. It’s not so much an issue for the x-axis, which we hold constant in this case, but may come into play with the y-axis…

It’s important to note that config.lua treats width and height as integers. This isn’t documented anywhere, but is easy enough to test: just “request” a content width of say 321.4 or a content height of 478.2 in config.lua, then report in main.lua what your actual values are for display.contentWidth|Height – they’ll be integers. display.actualContentWidth|Height are where you need to turn for the final resulting and potentially fractional* values.

So, the use of math.floor() on the calculated height assures that if the aspect ratio math doesn’t work out to an even integer (as is common with 4:3, 5:3, 16:9 or any other aspect ratios with repeating decimals) then we’ll always specify “just under” the exact fractional height, which will cause a “tiny bit” of vertical letterboxing.

(btw, all of this works just as well if the calculated height does happen to work out to be an even integer on devices with simpler aspect ratios such as 3:2)

Any residual fractional-pixel letterboxing will manifest itself as a difference between display.contentHeight and display.actualContentHeight. What we don’t want is any of that residual fractional-pixel letterbox height affecting the pixel alignment, and by default yAlign=”center” will split that fraction among equal top and bottom letterboxes*, causing a non-zero display.screenOriginY.

By using yAlign=”top” it will force any residual fractional-pixel letterboxing to all occur at the bottom of the screen, maintaining display.screenOriginY=0, and preserving the origin of content-versus-device pixel alignment.

* Maybe in a subsequent post I’ll delve into some other “weirdness” that crops up with Corona SDK, perhaps due to internal precision or rounding issues, if you ever attempt to “fact-check” the various display.* metrics with actual hand-calculated math. Things just don’t add up right. It’s definitely relevant, and factors into the solution presented here, but it’s pretty esoteric stuff. Suffice to say there are sufficient reasons for preferring “left”/”top” alignment here.

display.contentWidth = display.actualContentWidth = target width of 160, and is already pixel-perfect on a wide-variety of devices with 160N widths (where N is an integer), fe: 320×480, 480×854, 640×960, 800×1280

640x results, where retro-pixels are all consistently 4×4 hardware pixels (cropped vertically just to save space) click for full-size:

Will have apparent non-square pixels when nearest-neighbor sampling is applied on devices with width that are not 160N, fe: 400×800, 720×1280, 768×1024

720x results, where retro-pixels are inconsistenly 4×4, 4×5, 5×4, 5×5 hardware pixels (cropped vertically just to save space) click for full-size:

However, all is not as bad as it appears. Unless you’re actually using something like this QA pixel grid, which is intended to highlight problems, it’s actually quite difficult to see the inconsistent retro-pixels on-device once the hardware resolution approaches 800 or so (depending on display size, thus DPI). Pixel density beyond that tends to obscure these tiny irregularites.

And since many of the more-popular lower-resolution devices (below 800, with low DPI’s) are covered by the 160N resolutions, perhaps the irregular pixels might be something you could just ignore.

The Complex Approach

I know what you’re saying: “But I don’t want any non-square retro-pixels, ever!” Well then, something else is going to have to give, because 160 is simply not a common divisor among all the various devices.* That’s what the complex solution addresses.

* I believe that since the odd 1125×2436 iPhone X, the greatest common divisor across all various device widths is now 1. I have no desire to use UV coordinates as content dimensions – but try it if you like, it’s “fun”.

The difference between the approaches is that instead of keeping 160 as a fixed width as the simple approach did, the complex approach will allow a “160-or-a-bit-more” width, in order to obtain an integer content scale ratio against the device pixels.

This implies that the rest of the actual app’s code must be “flexible enough” to deal with the varying content width (as well as height). This is the trade-off, and may or may not present further issues. It’s really no more complicated than “traditional” letterboxing concerns, though the “source” of the letterboxed dimensions are under our control.

For example, if the app presents a scrolling world, then it probably won’t present much of an issue, just allow a bigger “viewport”. But if the app “depends” in one way or another on a known/fixed width, then it’ll either have to adapt to it or “ignore” (render useless) the extra area. (more on identifying that extra area later)

Remember that there’s really no concern about height – that’ll still be calculated from the device aspect ratio. The consideration can be limited to device width…

Consider the 768×1024 iPad. It would be possible to get perfectly square retro-pixels with 4×4 hardware-pixels if content width were specified as 192. (768/192=4).

Similarly, a 720×1280 device could achieve perfectly square retro-pixels with 4×4 hardware-pixels if content width were specified as 180. (720/180=4)

Similarly, a 1080×1920 device could achieve perfectly square retro-pixels with 5×5 hardware-pixels if content width were specified as 216. (1080/216=5)

Do you see a strategy emerging here? Essentially we’re looking for the greatest integer divisor of device width that will produce an integer quotient >= 160 for content width.

That sounds pretty simple to implement, and indeed it is. (drum roll please, this is what you’ve probably been waiting for,.. finally!):

Note certain similarities with the simple version – namely that we use “left”/”top” alignment, and calculate the height based on the device’s aspect ratio. The reasons why remain the same – BPBM.

So, what are the practical results of using this config on a non-160N device width? You’ll end up with a content width that is slightly larger than 160, but (this part is critically important, so I hope you’re still awake) the content width that you end up with is guaranteed to be an integer divisor of device width.

The extra content width needed to obtain an integer divisor is now a sort of “virtual letterboxed width” (as opposed to the actual letterboxing of height that was calculated). So, again, repeating myself a bit, the app will need to find something to do with that extra width.

For example, on a 720×1280 device, try this code which mimics the simple example above (and demonstrates how you might account for the extra width if needed):

Or grab the entire source code for this example.

Actual results on a 720×1280 device (cropped vertically just to save space) click for full-size:

The Hybrid Approach

“What?!?! I though you said there were only going to be two!! I am sooo sick of reading all this, just give me a one-size-fits-all answer and be done with it already!”

Whoa, whoa, settle down! This section doesn’t present anything truly new. Rather it just considers a possible hybrid of the prior two approaches taken together.

Why? The worst-case scenario that I’m aware of for the complex approach, based on extant actual devices, is the iPad Pro’s 2048×2732 display. The greatest integer divisor that can be used is 8, giving a content width of 256. So that’s 96 additional content units of width to deal with – quite a bit (+60%), considering the initial desire for just 160 width.

And maybe that’s just too much.

So, based on the discussion of high-DPI devices above (in the simple approach section) maybe you’d rather use something like the simple approach in that case. Or, craft a “custom” divisor of your own choosing to reduce all that extra width. The trade-off being that you’ll lose perfectly square retro-pixels.

That is, for such high-DPI devices maybe you’d be willing to switch back to non-perfectly-square retro-pixels just to avoid such an overly-wide content area. For example, use a divisor of 12 (giving 2048/12=170.666 content width) and with xAlign=”left” you could simply ignore any fractional pixels left-over on the right side,… right?

Almost, except that this will not work exactly. Why? Because, as stated earlier, width and height in config.lua are treated as integers. There was a reason we were looking for integer quotients as well as integer divisors. So the best you could do would be to specify a content width of 171. And if you use a non-exact integer width, then the integer height you calculate from it can be expected to be similarly non-exact, follow? (use math.ceil() on width, unlike math.floor() when calculating height – just trust me on this, or experiment and prove it to yourself – it’s too esoteric to get into here, but it makes for a better approximation of the letterbox math when using integers)

That non-exact width means that, if you then reverse the math to check yourself, you won’t back a perfect integer divisor of 12. Rather, the divisor using these sample values would be 2048/171~=11.976. So you’ll still be close to an integer divisor, but not quite – probably closer than with the simple method alone, and the closer you are to integer the fewer non-square retro-pixels will result overall.

Test yourself: can you discern the 11×12 non-square pixels below, even with a grid to help? (such columns ought to occur 4 times) On-device results (cropped vertically just to save space) click for full-size:

Let’s be honest – it’s not pixel perfect, and that WAS the topic, right? So why bother? Because mobile device development is often just a series of compromises. If a few percent of the pixels are a few percent anamorphic, that just might be good enough for some uses if it solved some other problem.

So, let’s table all of the results from the complex approach’s math (at least for known common device resolutions), allowing certain specific values to be overridden when desired, and falling back on the simple approach calculations for everything else not specifically covered.

Or grab the entire source code for this example.

Note that I’m not suggesting that this approach would be any better or worse than either of the two approaches alone, I’m just throwing it out there for consideration. It might benefit some use cases, but be of no value to others.

BTW, it should be obvious for the nice 160N width devices that all of the three approaches are equivalent. That is, if the device dimensions are such that an integer pixel-perfect scale is readily achievable as is, then that is what is used. It’s only for the “weird” resolutions that extra considerations might need to come into play.

Final Thoughts

If you’ve made it this far, congratulations! I honestly cannot fathom how you survived it. That’s probably the longest blog entry I’ve ever written, or am ever likely to write. Hopefully you’ve picked up a trick or two that’ll be of benefit to your specific usage. As for me, I have just now developed chronic carpal tunnel syndrome and will suffer in agony for eternity,.. probably,.. maybe,.. you’re welcome.

Did you find this useful? Want to support future efforts? Feel free to give my apps some free promo, or make a contribution (it’ll go towards a good cause: more LEGO for my kids). Thanks!