Mostly parsing and magics

Let me start this post with our usual screenshot!

As you may see, there are a lot of changes and improvements since my last post. I'll tell you the story of the last week in the followings, so prepare for a smaller wall of text, a lot had happened! ;)

The first and most obvious thing is that I took many ScummVM members' advices and doubled the screen height again. We all agreed that it will be it's final size. The algorithm was pretty easy, I only had to update Graphics::refreshScreen() a bit. (Oh, and I renamed the class Graph to Graphics. It was sev's idea and I agree with him. Now it reflects much more the purpose of the class, and I added so much things to it during the past days that it lost all of its connections to the Graph unit in Pascal, so there's really no point in keeping the old name.) So, the new refreshScreen():

void Graphics::refreshScreen() {
 // These cycles are for doubling the screen height.
 ::Graphics::Surface picture;
 picture.create(kScreenWidth, kScreenHeight * 2, ::Graphics::PixelFormat::createFormatCLUT8());
 for (uint16 y = 0; y < picture.h / 2; y++)
  for (uint16 x = 0; x < picture.w; x++)
   for (byte j = 0; j < 2; j++) 
    *(byte *)picture.getBasePtr(x, y * 2 + j) = *(byte *)_surface.getBasePtr(x, y); 

 // Now we copy the stretched picture to the screen.
 g_system->copyRectToScreen(picture.pixels, picture.pitch, 0, 0, kScreenWidth, kScreenHeight * 2);
 g_system->updateScreen();

 picture.free();
}

I think the comments make it pretty understandable. I still work with the original 640x200 resolution when loading pictures to the screen (If you check out graphics.h, you can see it for yourself in the form of kScreenWidth and kScreenHeight.), but when I show the actual output, I draw every line twice, so it's like we have a screen height of 400.

I also want to mention that I reworked loading of pictures a little bit in the form of loadPictureGraphic() and loadPictureRow(). The names of these functions came from here, referring to their behavior. Obviously the sooner one loads graphical data that uses "graphic-planar EGA data" as input and the latter one is for loading "row-planar EGA data". In the end, both produce a Surface object with the same format as output. This revision was needed because I had to use these two methods frequently and it made the whole thing much convenient since now Graphics::drawPicture() only accepts a Surface object and two coordinates, and draws the input picture (coming from one of the loading functions or simply by copying parts of the already drawn screen) to the given place. You can check out the current source code of these algorithms in graphics.cpp. I wouldn't describe them further here, because I talked about their operation in my previous posts a lot.

The next thing I spent a large amount of time with during the week was parsing. Since the original code was a bit hard to understand it took me more time to look through all of it than the actual coding, but finally it's done and working! :)
I introduced a new class, Parser (you can check this out here and here), which is practically a rework of Basher. One of my further plans is to completely get rid of Basher and move everything from it to Parser after renaming them. The "funny" thing is that the actual parsing is not done here (yet), but in Acci::parse(), and Acci::do_that() takes actions based on the result of the parsing (run animations, move characters, etc.). Parser now is only for processing the keyboard input and the handling of the visualization of it in a little command line under the main picture and over the toolbar. (You can see it in the screenshot at the top of the post, I wrote "get up" into it.) It's also my plan to move at least these two functions from Acci to Parser, and even more data, if I find out that they are only connected to parsing.
I wouldn't insert any code samples connected to parsing here now, since parse.h and parse.cpp contains quite readable code in my opinion, and Acci::parse() and Acci::do_that() is in such a bad condition that I don't want to advertise them at all. (:D) Let it be enough that they are called by Parser::handleReturn(), and they work properly. But a certain rework will be needed soon if I want to keep my code readable... Anyway, if you want to check out them in their current forms, here you go.

Also, not only the text input is working now (By using that you can wake up Avvy typing in "wake" or "wake up" and get him out of bed typing "stand", "stand up", or "get up". Other commands may don't work properly or don't work at all at the moment.), but you can move Avvy around in the first room of the game using the cursor keys or the home/end/page up/page down buttons executing diagonal movements with them. You can also control him with the numeric keyboard and stop him by hitting the numeric key '5' or hitting the direction key he is facing right now again.

We arrived to the second greater part of my last week's work: the MAGICS!
No, not turning people into toads or summoning a demon from Burning Hell using ScummVM, nothing like that. It's the name of the system in Avalanche which forbids the characters to cross the border of the screen and walk right on the toolbar for example. Let me illustrate it with the help of two screenshots!

As you may have noticed checking out Graphics, I added a new Surface object to the engine named _magics. This is the one which stores the image to the left. The white lines on it means the edges of the screen what Avvy (or any NPC) cannot cross.

They are drawn to this Surface in Lucerna::draw_also_lines(), using the data stored in Gyro::lines[] - which's content is loaded in Lucerna::load_also() amongst many other room-related data. If you'd like to read more about this matter, you can always check out the wiki pages of the original author.
So basically the algorithm in triptype::walk() is the following: we check if the next step of the character would cause him to cross a line on _magics and if it would, we simply stop the character.
Vigilant readers may have noticed the blue colored rectangle on the bottom left of the pictures, where the door is. It marks a different kind of "magic", which means that if we cross these lines, we want to leave the room.

My plans for the following weeks are making the speech bubbles (the so-called scrolls), the drop-down menu on the top and the transportation between rooms work. The drawing system will also need a certain rework, since now it can handle only one sprite per screen.

Avalot lives!

Yes, he does!

As you may have noticed, the menu bar on the top is also visible. So I can say that everything is shown what is needed for the first scene of the game! (Expect the clock and some status leds to the left of it. That will remain for later as I mentioned before.) Oh, and I got rid of that disgusting pink blanket as well and gave it the pleasant color of flesh! ;)

Let me speak first about Avvy, since it was a bit tricky to make him (and consistently the other characters during the game) appear, and I would have certainly gone mad if fuzzie didn't help me with a lot of explanations and suggestions.

So I added a new function to Graph: drawSprite(), which is - obviously - responsible for drawing the sprites of a character. The data of all characters are stored in an array named 'tr' located in Trip, which have the type of triptype. I made a little revision of this. The original code didn't divide data about a character in any ways, so I did. Everything connected to the display of a sprite (actually, the data what drawSprite() uses) went to a new type named SpriteInfo in grap.h and triptype got a new data member with the name of _info and the type of SpriteInfo. (You can take a look at these types and the array named 'tr' in Trip and Graph.)

The next thing was to implement Graph::drawSprite(). Here came in handy the new type I introduced, because now we only have to pass this data member of a triptype variable to the function and it can work using only these informations. Here it is:

void Graph::drawSprite(const SpriteInfo &sprite, byte picnum, int16 x, int16 y) {

 /* First we make the pixels of the spirte blank. */
 for (byte qay = 0; qay < sprite.yl; qay++) {
  byte *mask = new byte[sprite.xl];

  for (byte qax = 0; qax < sprite.xl; qax++) {
   byte count = qax / 8;
   mask[qax] = ((*sprite.sil[picnum])[qay][count] >> ((7 - qax % 8)) & 1);
   if (mask[qax] == 0)
    *getPixel(x + qax, y + qay) = 0;
  }

  delete[] mask;
 }

 /* Then we draw the picture to the blank places. */
 uint16 i = 0; // Because the original siltype starts at 5!!! See Graph.h for definition.

 for (byte qay = 0; qay < sprite.yl; qay++)
  for (int8 plane = 3; plane >= 0; plane--) // The planes are in the opposite way.
   for (uint16 qax = 0; qax  < sprite.xl; qax += 8) {
    byte pixel = (*sprite.mani[picnum])[i++];
    for (byte bit = 0; bit < 8; bit++) {
     byte pixelBit = (pixel >> bit) & 1;
     *getPixel(x + qax + 7 - bit, y + qay) += (pixelBit << plane);
    } 
   }
}

Use of image masks

I think the comments and mostly the code speak for themselves, but here's a brief summary of it:

The method consists of two parts: before the drawing, we have to blank the pixels of the sprite to black (0). Here, we use the array 'sil' which contains the mask for it. After that we can go through the image itself to draw it plane-by-plane, which is identical to the method I showed you before in my previous posts. You can read about the topic of image masks here in a bit more depth. Note that x and y here are the coordinates of the upper left corner of the rectangle which is an invisible frame around the sprite. (That was what you saw in my previous post in the color of magenta.)

The next thing I implemented was the changing of the disturbing pink color to a smoother 'flesh color'. It wasn't a big deal, I only had to set two palette entries what I did in Graph::flesh_colours(). (Keeping the name of the original function, but moving it from Trip to Graph.)

The third thing was the menu on the top of the screen. I have to tell you, it doesn't operate at all yet, it's only drawn to the screen so we don't have a blank line there. But of course it wasn't as easy as a bunch of writelns or couts. I had to draw the subtitles as pictures to the screen and they had a very nasty way of working. It's all done in Dropdown::chalk():

void Dropdown::chalk(int16 x, int16 y, char t, Common::String z, bool valid) {
 byte ander;
 if (valid)
  ander = 255;
 else
  ander = 170;

 for (byte fv = 0; fv < z.size(); fv++)
  for (byte ff = 0; ff < 8; ff++) {
   byte pixel = ~(_vm->_gyro.little[z[fv]][ff] & ander); // Note that it's the bitwise NOT operator!
   for (byte bit = 0; bit < 8; bit++) {
    byte pixelBit = (pixel >> bit) & 1;
    *_vm->_graph.getPixel(x * 8 + fv * 8 + 7 - bit, y + ff) = pixelBit + (pixelBit << 1) + (pixelBit << 2);
    // We don't have to bother with the planes, since they all have the same value. See the original.
    // Note that it's the bitwise OR operator!
   }
  }

 if (! z.contains(t))
  return;
 else {
  byte fv;
  for (fv = 0; z[fv] != t; fv++); // Search for the character in the string.
 
  // Similar to the cycle before.
  byte pixel = ~ ander;
  for (byte bit = 0; bit < 8; bit++) {
   byte pixelBit = (pixel >> bit) & 1;
   *_vm->_graph.getPixel(x * 8 + fv * 8 + 7 - bit, y + 8) = pixelBit | (pixelBit << 1) | (pixelBit << 2);
  }
 }

 _vm->_lucerna.blitfix();
}

At least here I didn't have to blank out the pixels or bother with planes and these made the whole thing a little bit easier. (In truth understanding the original Pacal code was much harder then implementing it in C++ again.) There is two stances of a menu item: valid (black) and invalid (grayed out). In truth, there's a third one: highlighted, but since the menu doesn't operate yet, I didn't have to bother with the implementation of it. Depending on the current state of the menu item, we get the pixels of the characters with bitwise operations. According to the original code, the planes all have the same value (since in dropdown.pas chalk() does the same for every plane), and since we have 1 byte for every row of a character in 'little', all the characters are 8 pixels wide as well. After that comes the underlining of a character of every text with a very similar algorithm and that's it: we are done. We call chalk() for every menu item after drawing a gray bar on the top of the screen and finally everything looks fine.

Implementing the drop-down menu system will remain for after midterm. My main concerns now are to make Avvy move around in the room with the help of the arrow keys, and if possible, put him into the bed at the beginning of the game, and make the speech bubble system work along with text input, so I'll be able to display the first few animations of the game (when Avalot wakes up to the screaming of her wife). I hope I'll be able to do all of this before midterm. I am giving it all I've got, and we'll soon see the result of it.

Loading pictures

Yes, loads of them! Let me show you the events of the past few days in the form of a screenshot first, and then I'll wrote about them in detail. Here you go:

The first thing you may have noticed looking the picture is that I changed back the resolution to 640x200. I decided to do so because I got the information that DOSBox only emulates the bigger screen height, and as Thomas said in a comment under my previous post, Avalanche really uses 200 pixel height, so I don't see the point in mimicking DOSBox's output instead of recreating the original experience. Also, if we decide that a bigger resolution would be nicer, it wouldn't take much more than adding a few lines of code to Graph::refreshScreen().

The second one is that there are much more things in the picture. Yes, I can say that the toolbar on the bottom is almost fully done and only the clock is missing from it. (Which - I assume - will miss for a couple of days/weeks because it's not so important regrading the actual gameplay, but tricky enough to recreate. And now I have much more urgent tasks to deal with before midterm - the characters, the dropdown menu on the top and the keyboard/mouse input -, and the days are just getting shorter and shorter.)
To be honest, after understanding how EGA graphics work and implementing the loading of the background image, the toolbar itself wasn't such a big problem, since it uses an almost identical method to load the parts of it as pictures. I can divide the bottom of the screen to 5 bigger parts: the background of the toolbar (the big blue thing), the "Thinks:" field of it, the score display, the clock, and that little picture (in the above screenshot a brown STOP sign) at the top left of it which tells the player if Avalot is standing in one place or moving in a given direction.
I implemented these parts in the exact order as I enumerated them in my previous sentence. To do this, I had to find out (obviously, with the help of fuzzie) that these pictures' encoding differ a bit from the background's. They are using row-planar EGA data, instead of graphic-planar EGA data which I used in the algorithm published in my previous post. Luckily, the methods for these two styles don't differ much, so the implementation of a function which will handle all of the pictures what are not the background was quite easy. To do so, I reworked Graph a little bit, and finally Graph::drawPicture() was born (to replace Graph::copySurface()):

void Graph::drawPicture(const byte *source, uint16 destX, uint16 destY) {
 // The height and the width are stored in 2-2 bytes. We have to add 1 to each because Pascal stores the value of them -1.
 uint16 pictureWidth = READ_LE_UINT16(source) + 1;
 uint16 pictureHeight = READ_LE_UINT16(source + 2) + 1;

 uint32 i = 4;

 Graphics::Surface picture; // We make a Surface object for the picture itself.

 picture.create(pictureWidth, pictureHeight, Graphics::PixelFormat::createFormatCLUT8());

 // Produce the picture.
 for (byte y = 0; y < pictureHeight; y++)
  for (int8 plane = 3; plane >= 0; plane--) // The planes are in the opposite way.
   for (uint16 x = 0; x < pictureWidth; x += 8) {
    byte pixel = source[i++];
    for (byte i = 0; i < 8; i++) {
     byte pixelBit = (pixel >> i) & 1;
     *(byte *)picture.getBasePtr(x + 7 - i, y) += (pixelBit << plane);
    } 
   }

 // Copy the picture to a given place on the screen.
 for (uint16 y = 0; y < picture.h; y++)
  for (uint16 x = 0; x < picture.w; x++)
   *(byte *)_surface.getBasePtr(x + destX, y + destY) = *(byte *)picture.getBasePtr(x, y);  
}

During the implementation of it, I found this little help during a simple search with Google. You can see that my code dips a lot from it: I read in the width and the height of the picture first, then create a Graphics::Surface object to store the picture in it, and then a very familiar algorithm comes with a little modification regarding the order of the planes. At last, I copy the image to a given point of the screen. So Tom Swigle, if you are reading this post somewhere, sometime: thank you, you are awesome!
After I finished with drawPicture, my only task was to replace the original PutImage() calls of the Pascal code with it, since I've done the loading of the pictures into byte arrays (what drawPicture gets as a parameter to work with) a few weeks earlier. And that's all, you can see yourself the result of it in the image above.

The last thing you might have noticed in the screenshot is a little pink rectangle in the middle of the screen. Don't worry, it's not a glitch! It's the very place where the hero of the game, Avalot d' Argent himself will manifest - I hope - within a couple of days. (I just put it there to see if I get the coordinates of the character accurately.)
Maybe next time we can see him in the screen as well? ;)

First screenshot

Yes, you are not dreaming! It's true, it's fresh, it's happening RIGHT NOW! Ladies and gentlemen, I proudly present you the very first screenshot of Avalot d' Argent in the holy embrace of SvummVM!

Okay, the colors really need some adjustments, but apart from that you can see that the background image is successfully loaded and shown accurately. (Of course here will come the GUI in the top and the bottom, but that's not my first priority at the moment.)

Lucerna

Let me tell you a little story about my fight with the bits! Everything started after my latest post: I began to mess around with graphics, and I had some mailing with dreammaster, who helped me a lot and gave me an awesome "Graphics in ScummVM for dummies" guide. By following that, I successfully set up the basics of the screen handling in my engine, what you can find here and here. Graph's initial purpose was to replace the Graph unit from Pascal with a C++ class, so it can work in synergy with Lucerna (the class/unit which is responsible for the screen, keyboard and mouse handling) but as time passes I am slowly getting convinced that it will eventually become a replacement for Lucerna. I don't see it clear yet, time will tell...

At the moment Graph is not much more than a wrapper around a Graphics::Surface object.

I got the resolution of the game (what you can find in the header file of Graph) from this function call in the original code: 

 gd:=3; gm:=0; initgraph(gd,gm,'');

Where 3 stands for EGA graphic driver and 0 means EGALo mode, which is equal to 640x200 screen resolution, 16 colors and 4 screen pages. (I'll bother with these "pages" later.)

I also had some difficulties setting up the color palette to mimic the EGA display, but dreammaster told me a lot about it, and by "borrowing" a little bit of code from Strangerke's Mortvielle engine, I get over with it quite fast too. As you may see it in the screenshot at the beginning of the post, there is still a couple of things to do with it, since the colors in the original game are much more livid, but I'll leave it's fix for later.

The next big deal came right after that. I had a long conversation with fuzzie (who is "substituting" Strangerke, my mentor, while he is on holiday) about the topic and she told me a great deal about EGA displays and how did they handle data.

The function which is responsible for loading the data of each screen is Lucerna::load(). The tricky part was here to notice the loop in the original code (which is fuzzie's merit), and recognize that we have to face the "four plane style". After that, recreating it in C++ was quite easy, and fuzzie helped me a lot with that as well.

Here's the result of it:

f.seek(177);

/*for (bit = 0; bit <= 3; bit++) {
 port[0x3c4] = 2;
 port[0x3ce] = 4;
 port[0x3c5] = 1 << bit;
 port[0x3cf] = bit;
 blockread(f, a0, 12080);
 move(a0, a1, 12080);
}*/

Graphics::Surface background;
background.create(_vm->_graph._screenWidth, _vm->_graph._screenHeight, Graphics::PixelFormat::createFormatCLUT8());

byte backgroundHeight = 8 * 12080 / _vm->_graph._screenWidth; // With 640 width it's 151
// The 8 = number of bits in a byte, and 12080 comes from the original code (see above)

for (byte plane = 0; plane < 4; plane++)
 for (uint16 y = 0; y < backgroundHeight; y++)
  for (uint16 x = 0; x < _vm->_graph._screenWidth; x += 8) {
   byte pixel = f.readByte();
   for (byte i = 0; i < 8; i++) {
    byte pixelBit = (pixel >> i) & 1;
    *(byte *)background.getBasePtr(x + 7 - i, y) += (pixelBit << plane);
   } 
  }

_vm->_graph.copySurface(background);

background.free();

As you can see I left the automatically converted loop in a comment. Under that, there's my C++ interpretation of it. You can find the idea behind that loop's core on this page. We guessed it's using four planes because the original loop goes from 0 to 3. So my code does the very same. It first reads the blue component of every pixel: since every bit represent a pixel, we jump in the x loop 8 times every time, and read a byte only after every 8th processed pixel. After that, we loop through the given byte, get the blue component of every pixel, and put them in their place. We do that four times, so every plane (blue, green, red, intensity) is read by the end of the big, plane loop, and finally we got the background image!

The interesting part with this is the following: we read an image with the size of 640x151. That's ok, but after playing the game a couple of times using DOSBox, I clearly saw that it shows the background image (and obviously the whole game too) in a much bigger resolution. I found that the window's size the emulator uses is 640x400. It's very strange since the original Pascal code (as I mentioned above) uses half of that height. To mimic DOSBox's output, I put a little supplement into Graph::copySurface() which copies every line of the background 2 times. After that, my graphical output became almost the same with the one DOSBox produced, but I am still not sure if it's the right way of operation and I am still searching for traces of this "stretching" in the original code.

(P.s.: Thomas: if you read this post and have any clue or suggestion about this resolution-problem, I'd be very glad if you could share it with me in a comment or by mail! :) Is the original output of the game really in 640x400, and if it is, where the "trick" is located in the original code which makes it possible?)

End of file handling

Hey everybody!

Sorry for my long absence! The case is that we just renovated our house with my family during the weekend (starting with Thursday), so I didn't have much time to spend with coding. But as we finished with all the work around our house yesterday, here I am again, more tenacious than ever!
I also finished today with the file handling (at least with those parts what don't involve saving/loading current game stance, only loading screens, sprites, texts, etc.). Sometimes figuring out what the Pascal sources are intended to do and converting it to C++ code was a rather tricky thing. (For example the problem with Pascal-style arrays and strings. Especially strings.) But finally I came over it, and now it's done and working. At least I think it does. I made several tests to them, almost trying with all the functions and all their source files (mostly *.AVDs), but the real test will be using them in the final code. (You can check this link, if you'd like to know more about Avalanche engine's file types and other related topics. Written by the original author of the game himself!)
My plans for the following days are these: I'll follow the program flow with the help of the warnings I placed previously in the code during stubbing, and going one by one, I'll implement each and every function in the code. I think the real pain will come with the screen processing units, but I don't think I have anything to fear as long as I have as awesome helpers as fuzzie or dreammaster.
(Sadly, my mentor, Strangerke is on holiday and we can connect only rarely, but luckily SvummVM has a great community, so I don't think that I'll be suffering from the absence of help even for a minute.)

Lets begin with file handling!

Yes, lets go!

As I am finished with the stubbing at last (*Phew!*), I can finally start reworking the code and implementing the functions of the classes made during the past two weeks. My plans for today is filling the gaps in the main class Avalot and then following the execution path of the program to get an even better sight of the behavior of it. (Because during the stubbing of the engine I think I understand the connections between the original units pretty well, but there are still blind spots here and there.) After that, I'll jump right into file handling and keep working on it until everything is going fine and smooth.

See you later! I'll report you guys as soon as I progress significantly!

End of stubbing

I just wanted to tell you guys that I am finished with the stubbing. And I am totally exhausted. And I am finished with the stubbing.
I can't describe how relieved I feel now. :D