Breaking The 640 Kb Barrier

INTRODUCTION:

As you know, when you program any project in any 16 bit language, you are usually limited, in one way or another, to the dreaded 640Kb of memory limit. This is because of DOS's internal architecture for the most part. Microsoft for example offered QuickBASIC version 4.5. This was and still is today, one of the most popular compiler to ever hit the market. However, like other 16bit compilers, it was a victim of that 640Kb memory limit as well. Even back then, the need for more memory for the applications grew to a point where Microsoft just couldn't ignore. Well after a couple years, Microsoft released what they called the Microsoft QuickBASIC Professional Development System (P.D.S.) version 7.1. Shortly after, Visual Basic for DOS was also released along with it's WIndows counter part.

The P.D.S. and Visual Basic DOS Professional both offer a means to get more memory for your applications by using a technique called overlays. This technique revolutionized the DOS 16bit world by providing a means of exceeding by many folds, the normal computing capacity of standard 16 bit compilers. Although Microsoft wasn't the inventors of overlaying techniques, the fact that such techniques were finally available in a language the likes of the QuickBASIC family of compilers was excellent news to the BASIC developers. This is what I will talk about in this technique, how to use overlays to create bigger, better and more complete applications and games.

WHAT YOU CAN USE TO OVERLAY:

As far as 16 bit DOS BASIC compilers, there are only two of them that can use overlays. Any BASIC compiler that came out before the two listed here simple didn't support overlays.

  • Microsoft QuickBASIC Professional Development System Version 7.1 This was the very first Professional Level compiler to hit the market. This offered QuickBASIC developers a host of new features that weren't available before. Manely, Microsoft ACCESS 1.0 database support, and Overlays. It also featured source code to offer a semi standard mouse driven User Interface for applications built with the P.D.S.
  • Microsoft Visual BASIC for DOS Version 1.00 Professional Here is the very last DOS BASIC compiler ever produced by the software giant. Visual Basic for DOS came in both the Standard and Professional editions. Among other features, a visual User Interface designer was available which dramatically cut down the time it would normally take to design the visual aspects of an application. The professional version also offered Microsoft Access database support and of course, overlays.

WHAT ARE OVERLAYS EXACTLY:

Well Overlays is a technique where a programs takes part of itself (part of the executable file) and uses it as something like a swap file where different parts of the programs, loaded from different files, to and from that area of the file. Each overlay is a seperate file on disk, when the programs needs a particular overlay, it calls it, the overlay gets loaded into the swap space and the overlayed code is then executed.

All you need to remember is that you shouldn't think that everything you code in a program should be overlayed. You have to remember that overlays, because of the way the work, will cost performance points. Especially if you need to swap overlays frequently. For example, it wouldn't be a good idea to overlay code that gets called in a loop from the main program or from another overlay either. This would cause alot of overlay swapping and would make things way too slow especially if the overlayed code being called performs alot of operations or actions. If you need to call a given function in an overlay you might want to bring the loop that calls it into that same overlay as well.

THINKING AND PLANNING IN ADVANCE:

I believe the best way to do the thinking and planning is to illustrate things with an example. Let's say I made a game called Nemesis. I've modularized my source code so that it gives me the following list of files. You don't have to call your modules this way either, we all have our coding standards and naming conventions, this is but a suggestion of how I would name things and is aimed at clarity more than official naming and organizations.

  • NEMESIS.BAS The main module of the game where everything else can be called from. It would have the menu system, shared variables, and other "globally" defined things.
  • MENUSYS.BAS Pretty much self explanatory, this is where the menu system would get coded in along with the branching of the selected menu items into the other modules of the project.
  • FILES.BAS I decide to put any function that needs to open a file, load it's contents or write to the file into this module just to keep things isolated.
  • VIDEO.BAS All functions related to writing to the screen or reading contents from the screen would typically be written here including saving and restoring screens or screen areas.
  • SOUNDFX.BAS IF there's anything that you can hear in the game as far as sound effects goes, this is where it would be. From punching to jumping to yelling to any kind of noise you can imagine.
  • MUSIC.BAS This would be where the loading and playing of themed music would be coded in. Loading of the file can be a digital music creation, a MIDI file, whichever. The routines to play the music file would also be coded in this module.
  • LEVEL1.BAS - LEVEL4.BAS In these 4 modules the different levels of game play are implemented. Level 1 is the easiest and difficulty levels increase some from module to module.
  • FINALLY.BAS The ending of the game can have more than one part. You can end the game by either finishing all the levels or if you die then you get a very different game ending to watch.

Now with this list of files you might be wondering what am I going to do with those. Well when you know that you will be overlaying parts of these modules in advance, you can organize your code into modules that can be overlayed intelligently. I say intelligently because you can do it any way you can imagine. So how do I define intelligently? Simple, overlays should be swapped the least amount of times possible so the game remains as fast as it possibly can. Like I mentionned before in this technique, such things like calling a function in an overlayed module within a loop is not a good idea especially if that overlay is swapped by another overlay in between the calls in the calling loop. Also, Modules that are called by other modules, if possible, shouldn't be overlayed either. So what does that leave us with?

WHAT GOES WHERE IN OUR EXAMPLE:

In our example we can safely assume a few basic things. NEMESIS.BAS, MENUSYS.BAS, FILES.BAS, VIDEO.BAS, SOUNDFX.BAS and MUSIC.BAS are probably not good candidates for overlaying. As you can probably assume by their names alone, they are central sub and functions modules that probably get called from more than one place more than one time. NEMESIS.BAS also typically has all globally shared variables as well. LEVEL1.BAS through LEVEL10.bas and FINALLY.BAS are all independant files each level is independant of the other as far as coding goes so we can assume that they could be overlayed. Same goes for FINALLY.BAS which is indepedant of all levels.

Making these assumptions is easy for me in this example. You might want to look at your code in your own projects to see what is independant of other parts of your code and what can be overlayed or not. the reason I say this is because in my example, I planned ahead before modularizing my source files this way. If you didn't plan for overlaying you might need to break your code into modules like I did and place your code a bit differently in the different modules than what you did already. Organizing the code helps when the time comes to link things together and create the actual overlay modules.

HOW DO WE GO ABOUT CREATING THE MODULES:

Before you go thinking that it takes rocket science to create overlays In both QuickBASIC P.D.S. 7.1 and Visual Basic DOS Professional, it's important to know that there's only one place where you decided what gets overlayed. You manage the overlays where you call the LINK.EXE utility. You specify what modules are to be overlayed by putting their names in parenthesis. For example:

LINK Main+(First)+Second+(Third+Fourth)

What this example says is basically: Create an executable with Main an Second as the main executable, First will be in an overlay, Third and Fourth will be together in another overlay. It's really that simple, this is how you decide what goes in the main executable and what goes in which overlayed module. In our example above, let's say that I want LEVEL1 and LEVEL 2 in an overlay, LEVEL3 and LEVEL4 in another overlay and FINALLY in yet another overlay. The LINK command would look like this:

LINK NEMESIS+MENUSYS+FILES+VIDEO+SOUNDFX+MUSIC+(LEVEL1+LEVEL2)+(LEVEL3+LEVEL4)+(FINALLY)

When the linker reads these values, it will go about simply creating the main executable and the three overlays as specified by the command. The overlays are controlled at the module level as you can see here, this is why it's important to take the time to plan your modules ahead at least a little bit.

THINGS TO WATCH OUT FOR IN OVERLAYS:

Indeed, there are some things to watch out for when deciding how many overlays you'll want to use. The following is a basic list of things to consider when organizing your modules:

  • A maximum of 64 Overlayed Files allowed: I think it's save to assume that probably no one will ever reach this limit, let's just say that I mentioned it here because it does exist and I think it's important that you know of it's existence.
  • Each Overlay can be no more than 256Kb In Size: Here you'll need to take into consideration the size of your .OBJ files especially if your LINK command combines two or more obj modules into one overlay.
  • All Modules should be compiled with BC.EXE with the Same Parameters: Rare are the times where one module, in a given project, won't be compiled with the same compiler switches as all other modules in a given project however if this is the case, you won't be able to overlay the module that needs the different switches. Again this is a "important to know" notion at the planning at organizing phase of overlay development.

IN CONCLUSION:

Just remember that overlays were a patch solution to a memory boundary problem of the 16 bit architecture that DOS offered it's users. However, if you take the time to plan your overlays a bit, it can definitely prove to be a powerful solution if you want to make bigger applications that are usually allowed in a 16 bit operating system. Use it wisely.

Stephane Richards

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-Share Alike 2.5 License.