Archive for September, 2016


This post is part of a three-part series. Directly access each installment here

Part IPart IIPart III


In the last installment I illustrated the immediate difficulties game designers run into when they try to dynamically create text output that takes random game objects into account. Grammar gets in the way…

wtext1

Today I want to show you how I approached the problem to create my Grammar script package for Unity3D. To understand the approach and its solution, it is best to a look at a sentence in a more abstract form. When we have sentence like this

The orc picks up a sword.

You can break it down into its parts.

[Definite article] [subject] [verb] [indefinite article] [object]

We can now easily create thousands of sentences, using this exact structure by simply substituting its elements. We could, for example, say

Frodo grabs a dagger.

Different sentence—same sentence structure. The definite article is dropped here because Frodo is a proper name, the verb is now to grab, and the object has become a dagger. Not a whole lot different than the previous sentence, but the fact that the proper name doesn’t need an article is worth remembering.

Another example could look like this…

The orcs grab some swords.

Same structure, but a few more rules kicked into action. Because the orcs is a plural form, the verb form changes as well. The s from grabs has been dropped to reflect the plural. In addition, because swords is also a plural form, the correct indefinite article is now the word some. (It could be omitted altogether just as well, depending on your preference.)

The easiest way to accommodate these adjustments in dynamic text is to create an engine that can generate the necessary sentence elements by itself. In order to do that, it needs some information about the objects in question.

For that purpose, I am setting up a data structure to hold the most basic grammatically-relevant attributes. This GrammarAttr data structure contains info, like the name of the object, as well as its gender, a count and plural flag, a flag to indicate whether it’s a proper name, and a few others.

The name in the structure is using a special format. Because it is not enough to simply drop in the name itself, I needed a format that allows me to generate correct singular and plural names of an object. Since no rules exist for this in any language, it needs to be hard-coded. To make it work I am using a format that consists of the word base, then the extension for the singular form and then the extension for the plural form, all separated by a period. Here are some examples.

wom.an.en
orc..s

To generate the singular form for woman, I would therefore concatenate the word base wom with the singular ending an. Whereas the plural would consist of the word base and the ending en to create women.

Identically, when creating the singular form for orc, I would take the word base orc and add the singular extension to it, which is empty, so the word remains orc. To generate the plural, we concatenate orc and s.

It is a very simple and very effective process, really, that I’ve been using unchanged for over 20 years. To make thing more convenient, my implementation also lets you drop the periods, if you have a proper name, for example, that does not have a plural form, so Frodo could be simply entered in the system as Frodo, without any periods.

roatext2

With all the attributed found in the GrammarAttr data structure, we can now go about designing actual tags that we can use to mark up the source text. I decided to enclose all tags with square brackets. This keeps things readable and it allows me to parse tags in the string very easily with a simple Regular Expression.

Since I want to make sure these tags can be easily understood and will help the translation of text into other languages, I decided to use natural language for these tags. In order to dynamically generate sentences like the ones above, my source string would, therefore, look like this

[The-name] pick[s] up [a-name].

Immediately, I am sure, you grasp the elegance of this because I am certain that just by looking at it, you will instantly understand how it works. But, you may also notice some of the inherent ambiguity.

The sentence contains the tags [The-name] and [a-name]. But how does the program know, which is which? This is where the GrammarAttr data comes in. When parsing the text to generate the final output string, I will pass a list of the referenced objects into the function as a parameter. This means that I will have a GrammarAttr object for the orc, and one for the sword.

Now it becomes a simple matter of reference. In order to tell the software, which object is which, we simply extend the tag and write this instead.

[1.The-name] grab[s] [2.a-name].

As you can see, some of the tags have a prefix now—once again, separated by a period—creating a grammar context. At the beginning of the sentence, I set the context to the first object (the orc) so the software will generate the output The orc. Note how the capitalized T in The-name indicates that I want the output text to start with a capital letter as well.

The Grammar module will first read the reference, grab the attributes for the first object and extract the name. Then it will slap a definite article in front of it—provided it is not a proper name, which does not require an article—and generates the output.

Naturally, if we were to set that data attributes to indicate that we are dealing with multiple orcs through the use of the plural flag or the actual count variable, the program would generate the respective plural form of the name.

As we move on, we hit the word/tag combination grab[s]. Since we have previously set a grammatical context—we are referring to the first object in the list of GrammarAttr parameters—the program can now check to see if there is one orc or many. It will then add the letter s to the verb grab, depending on the status of that flag. This will therefore generate either the sentence fragments The orc grabs or The orcs grab.

The next tag has a prefix once again, creating a new context. We are now referring to the second GrammarAttr object in the parameter list, and we want it to print the name of that object, complete with an indefinite article.

Easy enough. Grab the attributes, generate the name, generate the correct article, depending on the name and the other attribute flags, and voilà, we have a full sentence that says

The orc grabs a sword.

As I pointed out before, by simply changing the attributes in the GrammarAttr data structure, we can now create sentences addressing thousands and thousands of different objects, if we wish, using the same source sentence, all showing up with the correct grammatical rules applied.

Granted, this is a very simple sentence and a very small example, but it illustrates the approach I am taking to generate grammar-aware text strings. If you join me next time, I will show you how other tags allow us to create much more complex sentences, and in a future post, I will also explain, how all this fits into making localization easy and safe. I’ll see you soon…


Now available as a script package in the Unity Asset Store

keyimage500

Facebooktwittergoogle_plusredditpinterestlinkedinmail

This post is part of a three-part series. Directly access each installment here

Part IPart IIPart III


One of the key elements in your toolbox when developing role-playing or adventure games is a smart text generation stage that allows you to intelligently create dynamic text strings on the fly so that you can embed item names, monster names, character names and other things right in the text. Simple enough, right? Well, perhaps not as you shall see.

Even in today’s world of high-end RPGs, we still frequently see text output such as this:

Sword taken!
Acquired item: Sword

daitext

I may be over-simplifying this right now for illustrative purposes, but these impersonal, one-fits-all text snippets are the result of over thirty years of trying to avoid one basic, underlying problem—grammar in text generation.

Dynamic text generation adds depth to your narrative

See, in order to keep things a bit more interesting, the designers could just as well have picked a different sentence and made it look like this

Samwise picks up the sword and gives it a quick look-over before stowing it away.

or at the very least, in the fashion of old text adventure games, add an article to the respective words.

You pick up a sword.

Naturally, the requirement for longer text changes with each game. Some clearly keep text short as not to get in the way of gameplay, but games that rely heavily on text are better served with more verbose string generation. It is much more in line with the narrative storytelling that classic role-playing games were striving for, and it would create a whole lot more depth, wouldn’t it? It would, no doubt, and it was one of the key ingredients that made the Realms of Arkania games such a rich and incredibly detailed experience. So why aren’t more developers doing it? Are they truly so afraid that people don’t like to read? Hardly. If any audience in the computer game world is willing to read voraciously, it is role-players and adventurers.

poetext

No, the reason is purely technical in nature. To illustrate my point, let’s assume for a moment that Samwise is not picking up a sword but some coins instead. In this instance, the text output would have to look like this…

Samwise picks up the coins and gives them a quick look-over before stowing them away.

As you can see, the sentence has changed. Slight as the change might be, it is significant. Because coins are plural, the sentence needs to reflect that, resulting in different pronouns (them as opposed to it).

Things get even trickier when we change the sentence to something more complex like this

Samwise watches an orc pick up a sword, as he gives it a quick look-over before stowing it away under Sam’s watchful eye.
Pronouns and articles can wreak havoc on your sentences

There is, suddenly, a whole lot more grammar we have to deal with. “An orc,” and “a sword” are nouns that require indefinite articles, and as you can see there are different versions of it—“a” and “an,” depending on the word it is referring to. “It” is a pronoun referring to the sword that can change also if we are referring to a noun in its plural form. Then there is “Sam’s,” a genitive case of a name, which can also have different forms, not to mention that we need to distinguish whether we are referencing a general object like ”the orc’s eye” that requires an article, or that of a proper named object, like “Sam’s eye,” which doesn’t.

Here’s an example, how the same sentence changes if we just change some of the nouns it refers to.

The orcs watch an Elven girl pick up some coins, as she gives them a quick look-over before stowing them away under their watchful eyes.

Barely looking like the same sentence, in a way, but if you check closer, you will see that the sentence structure still remained the same.

I am sure you are getting the general idea of what I am trying to illustrate. Any time you make an attempt to dynamically create sentences that may contain any of a number of variable noun references, things can very quickly get rather complicated. You can’t just arbitrarily drop object names in the text and hope it all turns out right.

English is a fairly simple language

Funny enough, though, in English this may actually work quite frequently, especially when you stick with definite articles, but that’s only because English is a fairly simple and straight-forward language—grammatically speaking. When you get into Germanic, Roman, Slavic or Asian languages, all bets are off, because these languages have reams of grammar rules that do not even exist in English. Some feature arrays of pronouns that depend on the referring noun as well as the referred-to noun. Without a proper plan of attack, managing this kind of variety is a futile effort and will inevitably lead to a lot of code duplication and extra programming—or, God forbid, tons of specially written/translated sentences to capture different scenarios.

pttext

That is the very reason why many developers and designers will revert to the banal, one-fits-all text version I described in my opening. They are easy and safe cop-outs. Not much can go wrong when you essentially talk in bullet points. It’s just not very inspiring and certainly doesn’t help to transport you to another world. Nothing screams Middle-Earth, Tamriel or Azeroth quite like…

Sword taken!
Acquired item: Sword
Spider: 5 damage!
Druid misses Orc

…and neither do dialogues or setups in which everything is static and every step has been predefined without any consideration for character weaknesses or traits to add personality or twists.

A world dense with monsters, characters and items begs for more than banal responses

While developing the Realms of Arkania games, many years ago, the narrative depth was crucial to us. We wanted to make sure that we could easily adapt situations, plots, subplots and sentences to fit all sorts of situations. Those of you who played the games will certainly remember this. With a personalized party of up to seven characters, a world dense with monsters and encounters, and gameplay that featured many hundreds of items, a more strategic approach was necessary, especially because we had planned, from the beginning, to translate the game in different languages.

roatext

At the time, I developed a system that allowed us to make our text output grammar-aware but in all fairness, it was a rather clumsy, academic approach that was way too convoluted to be useful to anyone without the proper technical background. But it was a first step.

As I began designing Deathfire a few years ago, an RPG I planned on making, I approached the subject matter anew because the focus of the game was on the Psycho Engine, a piece of software that would allow me to dynamically adjust gameplay and the behavior of entities in the game on the fly. Therefore, I needed a maximum of versatility and flexibility, while also making sure the game could easily be translated into as many languages as possible. With a fresh mindset and a bit of development experience in Inform under my belt, I designed a new system.

Grammar awareness also makes translation much easier and safer…

Inform, as you may know, is a programming language specifically designed for interactive fiction/text adventures, and, as you would expect, these types of games have the exact same challenges to contend with. The key difference, however, between Inform and my grammar-aware text generation is that the Inform engine actually knows what you are referring to, because everything in the game world is defined in terms of addressable objects, whereas a generic dynamic text generation system is not aware of the actual context and needs to derive the necessary information from somewhere else.

…the grammar logic is in the text, not the programming!

I had never had the opportunity to actually implement the system—it was a spec design on paper, more or less—but in recent weeks, I’ve had some spare time and decided to write a C# implementation for Unity3D. With all of the above in mind, I sat down and created a code module that enables me to easily create dynamic sentences that are grammar-aware and that allow me to use natural language to define a grammatical context. In my next blog post, I will tell you the details, how it all looks like. Join me then…


Now available as a script package in the Unity Asset Store

keyimage500

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Alongside a new PS4 Slim model, last week Sony also announced a new PS4 Pro model to be released in November for $399. One of the key features of this mid-cycle hardware upgrade is that this new console will offer 4K ultra high definition resolution, along with the expanded colorspace of the ultra high definition format that allows for native HDR. Alas, according to Sony’s announcement, the PS4 Pro will not play 4K UHD discs.

ps4_1Now, a lot has been made of this lack of 4K UHD disc support over these past 24 hours and many fans are disappointed. Expectedly so, but… there is more to this story than you might suspect.

Before you dismiss the PS4 Pro out of hand, let me fill you in on a few things that you may not be aware of, because they have been hardly discussed in public so far.

The 4K UHD disc format is actually a standard Blu-Ray format. When the Blu-Ray specs were devised, it allowed for a large number of layers on a disc to accommodate the voracious, ever-growing storage needs of digital media. The format was designed with the future in mind. While, in practice, so far no real application has been out there, to the best of my knowledge, that used more than two or three layers, there are plenty of provisions in place to create discs with four, five, six or even more layers on a single disc. And it is this feature that is being used on 4K UHD discs. At heart, they are still only Blu-Ray discs, just with more storage, differently encoded video files on them and a slightly different menuing system that accommodates new features.

As a result, any Blu-Ray drive that properly follows the Blu-Ray specifications should theoretically be able to read these discs. (Note that I said read, not play.) Now, in the real world, because these additional layers have not been used before, there are, of course, low-cost drives out in the market that do not support the feature—a result of cost cutting measures. However, every modern Blu-Ray drive does well to integrate the capability to read additional layers. I am sure, most manufacturers have complied with those specs for some time now and I have absolutely no doubt that the drives found in the upcoming PS4 Pro will be able to read these high density Blu-Ray discs with absolutely no problem.

So, to be clear, from a technical standpoint, there is no difference between a regular Blu-Ray disc and a 4K UHD disc. In fact, they are authored the same way and they are replicated the same way. In fact, they are replicated on the exact same machines. It was one of the appeals that the 4K UHD format had for studios that existing production lines could be used to manufacture and replicate these next generation discs, making them very cheap to produce right out of the gates.

ps4_2

So, why doesn’t the PS4 Pro support 4K UHD discs? The answer to that may lie in Sony’s Consumer Electronics division. At this time, Sony is preparing the launch of a series of settop 4K UHD players. Naturally, the company wants these players to succeed. They want to sell them in quantities that makes the company proud. If they would add 4K UHD capabilities to the PS4 Pro, many people would simply forgo a settop player and use the PS4 Pro as their player of choice, the way they did when the PS3 first arrived, complete with Blu-Ray capabilities. Only this time they don’t have a format war to win (remember, Blu-Ray competed heavily against HD-DVD at the time and the PS3 was essentially the format’s secret weapon that decided the war), so there is no need for them to cannibalize their own divisions. When it comes right down to it, the lack of 4K UHD disc support is simply a business decision that actually makes sense for the company. Why? Because this is not the end of the story.

At this time, I have to admit that the PS3 is still my preferred Blu-Ray player of choice. The reason is simple. Unlike settop players, Sony made sure the PS3 always kept up with improvements and innovations in the Blu-Ray format. When DTS-HD Master Audio arrived, all they did was give users a firmware update and voilà, you could suddenly play DTS-HD Master Audio tracks. When Blu-Ray 3D came around, it took only a firmware upgrade for viewers to enjoy their 3D content. The list of capabilities they added via simple firmware updates was truly impressive throughout the years.

There is absolutely no doubt in my mind that we will see the same kind of commitment to unlocking features-as-you-go when it comes to the PS4 Pro and 4K UHD discs. When the company has safely shipped settop players to stores and has seen them sell through, you will find a firmware update on your PS4 that will suddenly enable 4K UHD disc playback.

Is it possible? It is not only possible. It is extremely likely. There is no technical limitation. The PS4 Pro is capable of decoding and displaying full 4K UHD video streams with HDR, and the console is capable of reading Blu-Ray discs. That is all the hardware capability you need to play 4K UHD discs. Everything else is merely software to facilitate the data transport from the disc to the display logic and to handle things such as disc menus etc, all of which follow established Blu-Ray specifications, only at a higher resolution and in an extended colorspace.

So, once again, before you dismiss the PS4 Pro as a failure and as a no-purchase because it doesn’t support 4K UHD discs, think again. This limitation will be only temporary and its solution will be a mere firmware update away.

With that in mind, perhaps it is time for you to stop over on Amazon’s website and pre-order your PS4 Pro?

Facebooktwittergoogle_plusredditpinterestlinkedinmail