/\ / \ ../ \....METALBASE release version 3.1...................................... / \ / \ Written August 1989 through October 1990 by Huan-Ti \ / \ / Initially released December 10th 1990 \ / \ / \/ "And Calvin ate. And Food Lion went out of business. Amen." --1st Calvin 4:8 WHY_METALBASE_?________________________________________________________________ The Amiga 500 multi-tasks. Wow. It has great graphics. Yeah. Phenomenal sound. Yippee. It costs $500. I bought one. A DBase package costs more than the computer. So does Paradox. And Informix. And almost every other database package on the market. I didn't buy one. But I needed a database for my system. Anybody who's ever worked with a database knows they're addictive--code that's written without a flexible database backing it up lacks something. And DBase wants $600 to put that something back into programs. No thanks. That's why MetalBase was written... its forerunners (versions 1.0 and 2.0, not available) were designed by the same author with the Amiga line's needs specifically in mind: relatively slow floppy drives (Compared to hard drives), so sparse read-head maneuvering was essential. Relatively small memory, when compared with the amounts of memory required by most C compilers (Lattice needs roughly 300k just to run, for example), so the code had to be consise, pure, and use few variables. Expansion was always available; hard drives, huge memory capacities, extra processors, ram drives -- so the system had to be readily expandible and flexible enough to meet the requirements of both a 512k, 1-drive machine and a 4-megabyte, 120-meg hard drive-driven setup. And they were. The problem was, Amigas can multi-task. So what is the rest of the computer doing while a database runs around looking up records on one drive? Nothing. Why not let it look up records on another drive? Fine... that works. What about on the same drive... using, say, a ram disk, so there are no real read-heads to worry about positioning? That would mean each relation would have to support multi-user access... a requirement around which MetalBase versions 3.0 and 3.1 focus. Not only can many terminals access many files at once, but on multi-terminal systems, each terminal can work simultaneously on any given file. Terminals can each use as many relations as a system will allow, and each relation can be used by over one hundred users at once. The code used to create MetaBase 3.X is completely compatible with the Unix ) operating system and all of its derivatives, so its inherent multi-user ability places it in firm working footing on the most advanced systems on the market. Relations are designed by a user to match his/her exact requirements, indicies placed as needed on single or composite fields, and records can be added, retrieved, changed, and deleted as any C programmer so desires. The interface is as simple as possible, very fast in execution, and provides a wide range of error types for exact specification of problems. A user on any terminal can choose to lock out other users from any number of relations, so data that is about to be changed will not be read by others before changes take place. Automatic temporary locks are placed on a relation which is about to undergo a deletion or addition, thus delaying other users' access to files, so that no data will be read until all indicies have been updated correctly. This process is entirely system-driven; users never need worry about it at all. If another user is adding a record to a relation, any terminal requesting a record lookup will simply wait until the addition is complete before proceeding. As records are deleted and added through normal use, the algorythms used to update indicies automatically keep them at top operating speeds, so any given relation's number of records can grow exponentially while causing only a linear increase in look-up time. Any relation can contain over four billion records (Nearly four billion more than most systems will ever need), and each record's length can be as large as its system's memory can handle. In perspective, MetalBase is infinitely expandible, while requiring very little of the system on which it operates. RELATION_DESIGN________________________________________________________________ A sample relation. Data Field # -> 1 2 3 4 5 is read by selecting one Record # _____ ___ ____ ___ ___________ record from the others by 1 |_____|___|____|___|___________| various criteria, and 2 |_____|___|____|___|___________| returning it to the user. 3 |Smith| 12| 114| A2|Houston, TX| Record 3 (In this example-- 4 |-----|---|----|---|-----------| actual MetalBase records need no such numbering system), if returned, would be placed in a string as "Smith| 12| 114| A2|Houston, TX|" A relation is a collection of records, each record consisting of the same number of fields, character strings whose length remains constant from record to record. Each field need not have the same length as the next in any given record, but each record will contain the same fields as the others. The data the fields contain is, of course, the main purpose of a database--when thousands of records exist, a system without a database would need vast amounts of time to find any given record. Databases decrease dramatically record retrieval time, while requiring as little additional time to add new and change old records as possible. The key to a database is its indicies--pointers arranged in such a fashion as to show the system where to look for the record for which it is searching. The more of these collections of pointers, or indicies, a database has, the greater the number of different fields a system can search through in order to find a record meeting certain criteria for a field (Or fields... a single index can be used to first sort records by one field, and also sort duplicates with respect to a second field, etc. MetalBase allows these composite indicies to key off of as many as 5 different fields). However, databases require time to update each index every time you add, delete, or change a field which has an index placed upon it. So although retrieval criteria possibilites are expanded by using many indicies, the amount of time required to change the existing records in a database will also increase. For this reason, the number of indicies a relation will have should be considered carefully. Use what is needed, but no more. Relations for MetalBase take the following form (Words in brackets are optional syntax, words enclosed in <> are sample entries): [relation] field [length] <5> [right] field <#_purchased> [length] <3> [] field [length] <4> [] field [length] <3> [] field [length] <11> [right] index [on] customer [] index [on] part_number|price_code [with duplicates] index [on] price_code [] [end] Note that the relation-building program released with MetalBase 3.1 also supports comments at the end of the "field" and "index" lines, as in: field shop_address length 11 ; Comment here ; The semicolons must not touch text on either side. MetalBase 3.1's build function also supports the ignored keyword "left" in place of "right", for those who put it there by instinct. The first line of a schema simply tells the computer what to call the new relation. The word 'relation' is optional, as is all text enclosed in brackets. The second section of a schema describes the fields of which every record in the relation will be composed. The word immediately following 'field' is simply a name for the field. The next point of interest is the number after 'length'... this number indicates the number of characters in that field. MetalBase will allow any positive integer here your computer can handle, and if any length, or the sum of all lengths, is indeed large, BUILD will prompt you to add certain lines to the beginning of your code to warn the system of excessive-length records. The word 'right' immediately following the field length is optional--if it exists, any data placed in the field which does not contain enough characters to fill the field will be right-justified. This is essential for number fields which will be indexed, as computers interperet "42" as a number which is much less than "7 " (Note the space to the right of the seven). Without the word 'right', all data will be left-justified in the field. After all fields have been declared, indicies are placed on fields and combinations of fields. MetalBase requires that you have at least one index in any relation (This is only logical--a database without indicies is nothing more than a text file). After the word 'index' in each line, the names of fields to be indexed appear, separated by vertical bars ("|"). If more than one field name appears, MetalBase will declare that index composite. Making indicies composite is encouraged--it adds no time to relation updates, but improves the appearance of output for nearly all purposes. Occasionally certain data should not be repeated... for instance, a person should not be able to visit a physicians office twice at the same time. In this case, were a relation defined as consisting of the fields "name" and "visit_time", an index would be declared on "name|time" WITHOUT including the words 'with duplicates', as seen above. If such a relation were built, MetalBase would not allow the space-time continuum to be stretched, as described previously. Including the words 'with duplicates' after the definition of an index remove this error-catching system, for that index alone. Use of the duplicate-catching system is entirely a case-to-case decision... there is little difference in the amount of time used between implementing and not implementing it. If the schema described previously exists as "/usr/joe/equip.s", the command BUILD, a utility included for use with MetalBase, will create a relation called "equipment.rel" under the directory "/usr/joe". A sample output of BUILD is as follows: % build /usr/joe/equip.s Building relation equipment under directory /usr/joe Fields___________________________________________________ customer [5] L-Just #_purchased [3] R-Just part_number [4] R-Just price_code [3] R-Just shop_address [11] L-Just Indicies_________________________________________________ customer..........................Duplicates allowed part_number|price_code............Duplicates not allowed price_code........................Duplicates allowed Do you wish to continue to build this relation [Y/n] ? Relation created--zero entries. % ls -C /usr/joe equip.s equip.rel % Once a relation has been built, it contains no entries, and is ready for use. BUILD's sole use is the creation of relations... other utilities (discussed later) must be used for maintenance of relations (if needed). IMPLEMENTAION__________________________________________________________________ A sample program implementing MetalBase follows: (*) #include main () { int file; char str [40]; (**) if ((file = mb_inc ("/usr/joe", 128)) != 0) { printf ("File '/usr/joe.rel' cannot be found.\n"); exit (1); }; if ((mb_sel (file, 2, str, EQUAL, "114|A2") != 0) { printf ("Record cannot be found.\n"); exit (1); }; printf ("Record contains : %s\n", str); mb_upd (file, "Su | 14|112|C3|Dallas, TX"); mb_sel (file, 1, str, CURRENT, ""); printf ("Updated to : %s\n", str); mb_rmv (file); } Manipulation of relations using MetalBase is simple... by including the MetalBase header file in your programs (As line (*), above) and compiling using the option "-lmb" (Or "library ..., mbase.lib, ...", depending on your compiler), you add extra commands to C. A summary of these follows: - mb_inc (filename, int) -- this command includes a relation on your desktop, so that it becomes accessable. Any relation you wish to work with must be included, as in line (**) above. If mb_inc returns a negative number (Zero is a valid return code-- it does not indicate an error), an error has occurred (The most likely one is that the relation cannot be found under the given directory). Note that "filename" does NOT include the ".rel" suffix. BUILD, in analog, does not require that you include the ".s" suffix on the name of the schema you are working with, but the schema file must end in ".s", as relations must end in ".rel". The integer passed in is the encryption key for the relation--see the section on encryption for more information. Note that MetalBase 3.0, as it does not support encryption, does not require this integer. Sorry for the inconvenience. - mb_sel (file, index, buffer, action, comparison_value) -- this command searches a given relation for the record you desire, described by "action" and "comparison_value". "file" is the number returned by mb_inc... this code is used in all other functions to indicate which relation you intend to work with. "index" is the index number by which you wish to search... numbered sequentially by the order in which they were defined in the schema, starting with the number 1. "buffer" is merely a string into which MetalBase can return the record, when found. Note that the ENTIRE record will be returned, not just the fields indicated through the schema for use by the index. A list of valid actions follows: FIRST (or FRST) -- searches for the first record alphabetically. LAST -- similarly, seeks the last record alphabetically. NEXT -- searches for the next sequential record alphabetically. PREVIOUS (PREV) -- as NEXT, searches for the previous record. EQUAL (EQUL) -- seeks a record containing fields exactly those in "comparison_value". If the index used allows duplicates and some exist, EQUAL returns the first of these: all duplicates can be found by subsequent "NEXT" searches. GTEQ -- EQUAL returns a NOT_FOUND error if a record cannot be found to precisely match the comparison. GTEQ acts as EQUAL in all respects but this; GTEQ will find the record greater alphabetically, or precisely equal to (If one exists), the comparison. GTHAN (GTHN) -- Acts as GTEQ, except GTHAN will ignore precise matches. Useful if you wish to begin searching names, say, after the "Smith"'s. Another way to accomplish this would be to use LTEQ + "Smith", followed by a NEXT... LTEQ -- Searches for a record less alphabetically, or equal to (if one exists), the comparison. In the event that duplicates of the record about to be returned exist ("Duplicate" being defined locally by a record, whose fields used by a given index are precisely equal to another record's such fields on the same index), LTEQ returns the last of these: all duplicates can be found by subsequent "PREVIOUS" searches. LTHN (LTHAN) -- Similar to LTEQ in all but one regard: LTHAN will not return a precise duplicate, instead searching to its left for the record before it. CURR (CURRENT) -- After a record is found by a search, MetalBase remembers the location of the record. Searching with the action CURRENT re-reads this most recent record, and returns it in its entirety. As with FIRST, LAST, NEXT and PREVIOUS, the value of "comparison_value" is irrelevant in CURRENT -- however, CURRENT is also oblivious to index used (As no actual searching takes place... MetalBase merely recalls a known record). Please note that the above-described action names merely stand for integers, and MetalBase will not understand if you use quotes around the names. These actions must also be written in upper-case. "comparison_value" is a string containing a record consisting only of the fields needed by an index... each field separated from the next by a vertical bar. The last field may optionally be terminated with a vertical bar, if desired. - mb_upd (file, new_rec, mode) -- After a record is found, it may be updated through mb_upd. MetalBase simply replaces the old record's fields with those indicated in "new_rec". This process usually requires more time than any other, as the record must be disconnected from each index in turn, changed, and reconnected. Occasionally, changing a record will not change certain indicies. When this occurs, the update would take less time if the index were skipped... and if you feel this extra time is necessary, type the word FAST for "mode". Like the actions for mb_sel, FAST and SLOW should not be surrounded in quotes. Disconnecting and reconnecting a record, to the contrary of the preceeding argument, is a very beneficial process, even for indicies which are not changed through a specific update. The algorhythm used for such updating is designed to help normalize the database, much as a kitchen-knife case which sharpens blades every time a knife is removed or inserted. A "normalized" index is one in which records have been packed as closely together as possible, minimizing lookup time. If "mode" is set to SLOW in mb_upd, ALL indicies will be updated in this way every time a record is updated. An update will be aborted if the changes would violate a non-duplicate index. - mb_del (file, verification) -- After a record is found, it may be removed completely by use of mb_del. Once a record is deleted thus, it CANNOT be retrieved. The "verification" argument should be NULL ("") -- its only purpose is to ensure that the command has not been confused with mb_rmv, described below. Due to C's handling of file-length limiting pointers, mb_del does not decrease the actual amount of storage space for a relation. Subsequent additions, however, will not increase allocation space for a relation until the number of records added exceeds the number previously deleted. To recover this space and physically remove deleted records, see the section on Utilities, below. - mb_add (file, new_rec) -- This command adds a new record to a relation. If adding the record described by "new_rec" would violate a non-duplicate index, the addition to the relation will be aborted and an error returned (See the section on Errors, below). - mb_lck (file) -- If, as an example, a program reads a record from a relation used by many terminals simulaneously, waits for a user to enter a new value for one of its fields, and updates the record, it could prove disastrous if the record was read by one terminal, then another... updated on one & re-written, and re-updated by the second terminal. To prevent this type of scenario, mb_lck, when used, excludes other users from including a relation on their desktop. Such a lock cannot be installed if more than one user is already using the relation. - mb_unl (file) -- Removes a lock placed on a relation by mv_lck. - mb_rmv (file) -- This command removes a relation from your desktop. If the file has been locked previously (Via mb_lck), the lock is removed. Note from the example program above that comparison values in mb_sel, and new_rec in mb_add and mb_upd need not be spaced correctly, or at all... MetalBase will automatically justify each field properly before implementing it. However, if "7.3 |" (Note the trailing space) is included as, say, a six- character right-justified field, it will be spaced to " 7.3 |"... the trailing space remains intact. Left-justified fields similarly retain any leading spaces. ERRORS_________________________________________________________________________ Build / Errors : Cannot open ............The schema indicated to Build cannot be found. File holds no schema...The schema indicated cannot be used definition as a definition of a relation. The most likely cause is that the file is empty. Field declared after....All fields must be declared before any indicies indicies are declared. Field declared twice....Field names must be unique. Spaces are not allowed in field names. No fields declared before.......A relation must contain fields, and these end reached must be declared before any indicies. Incorrect syntax................The definition of a field's length or justification has been entered incorrectly. Field undefined.........An index has been placed on a non- existant field. Check spelling. No fields declared before end...The schema definition is incomplete: it reached must contain at least one field and one index. No indicies declared before.....See the above error description. end reached Identifier not recognized...The only valid words for the beginning of a line are "relation" or a relation- name (As the first line), "field", "index", and (optionally) "end". Cannot open relation............The file cannot be written... possible causes are a media error or write-protection on the file. Relation is busy................Relations cannot be re-built until it is not being worked with by any users. Build / Warnings: Please re-define BUF_LEN as.....If a relation's record length is excessive, you will receive this warning. The line "define BUF_LEN xx", with the x's representing the number described in the warning line, must be included before "#include ". Please re-define MAX_FLDS as....If any relation contains more than 30 FIELDS (_not_ records), the line" #define MAX_FLDS xx" must be inserted at the beginning of your program, as with the previous warning. Please re-define MAX_CUT as.....If a relation's record length is greater than 80, you will need to insert the line "#define MAX_CUT xx", as with the previous warnings. Please re-define MAX_RPT as.....If the length of any field is greater than 240 characters, you will need to insert the line "#define MAX_RPT xx", as with the previous warnings. The file about to be created....If you build a relation where a relation already exists of the same name already exists, any data in the existing relation will be erased in favor of the new, empty relation. MetalBase / Errors : 0..OKAY...............This is the generic no-problem return code. All functions except mb_inc should return 0 if they have been able to function properly. -1000..NOT ENOUGH ROOM....Too many relations are open at once -- mb_inc cannot allocate enough space for the new relation. This error can be prevented by re-defining MAX_REL, as with build's warning messages, to a value greater than its default of 5. In addition, having many files open at once may require that you warn ... consult your system manuals. -1001..CANNOT OPEN........The file , where is the name RELATION passed to mb_inc, cannot be opened. In all probability, it does not exist. -1002..RELATION_HUNG......A user has stopped a program during a record addition, deletion, or update. These functions place temporary locks on a file, which are removed at the end of their respective procedures. Normally other functions will simply wait for the file to be unlocked, but if the wait time becomes excessive, this error is returned and the function aborted. Try the function again, and if you recieve this error a second time, see the section on Utilites to remove the lock manually. -1003..CANNOT READ........The file cannot be read. This error RELATION is returned only from mb_inc, and is probably caused by read-protection placed on the file. -1004..RELATION BUSY......No more than 120 users may work simultaneously with a given file. -1005..RELATION LOCKED....The relation about to be opened has been locked intentionally by another user, and is unavailable for use by others. If no one is using this relation, see the section on Utilites in order to remove the lock on the relation. -1006..LOCK DENIED........A relation cannot be locked while more than one user is working with it. If no other users are on the relation and you still receive this error, see the section on Utilities in order to clean the user-count on the relation. -1007..ILLEGAL DUPLICATE..The record that has been offered for addition/ update would violate an index that has been declared to not allow duplicates. -1008..CORRUPT INDEX......This relation has an error on an index. See the section on Utilites to repair it. Try to find which index you were working with at the time of the error, in order to speed repair. -1009..INVALID FILE CODE..Relation codes, the numbers returned by mb_inc, are always positive. This error is returned upon receipt of a negative relation code. -1010..NOT OPEN...........The relation with which you are trying to work has been closed. -1011..RECORD INVALID.....Not enough, or too many, fields have been given for update or addition of a record. -1012..INVALID COMP.......The comparison value submitted via mb_sel must contain exactly the number of fields required to complete a given index. -1013..UNKNOWN ACTION.....A very unlikely error, receipt of this code indicates you have passed a number greater than 12 or less than 0 to mb_sel as an action. -1014..NOT FOUND..........Generic code returned when the record described cannot be found in the relation searched. If you feel it should have been found, check the index number, action, and comparison value used in the search. -1015..NO CURRENT.........Update and delete act on the record which has RECORD been found most recently. When a relation has just been opened, and just after a record is deleted, there is no such record. -1016..INVALID INDEX......A bad index number has been passed to mb_sel. Valid numbers range 1 through the number of indicies in a relation. UTILITIES______________________________________________________________________ The supporting utilities to MetalBase have not been released with this package, primarily because I have not bothered to write them yet. See the section on SHAREWARE below to see how to register yourself with me (god), as I'll write any of the following upon request once you're registered. Utilities include: * Repair of bad indicies (Also normalizes relation for maximum speed) * Recovering lost space from previous deletes * Overhaul of relation (Last two, in one) * Removal user-placed locks * Removal system-placed locks ENCRYPTION______________________________________________________________________ MetalBase 3.1 (not 3.0) automatically encrypts all files. Users of version 3.0 may just as well skip this section. When a relation is opened (using mb_inc() ), an integer encryption-key is passed in to MetalBase. Any records which are added to that relation are encrypted using this key, and any records which are retrieved are decrypted similarly before they are returned. In this way, any information which is physically stored is encrypted beyond recognition. As long as this integer key is the same every time the relation is opened, the encryption is totally user- transparent: records are passed to mb_inc and mb_upd in normal text, and are retrieved in normal text, but are stored garbled. If the key changes, however, the records retreived in the second session will be decrypted under the new key, whereas they were encrypted under the former... and the output will be garbage. The values of the integer determine the level of encryption to use (note that similar numbers, say, 140 and 141, give very different encryption results): 0- 64 : Only minimal encryption (Due to the way the encryption method works, every character is simply 2's-complimented. This is exceedingly simple to decode, but is easily sufficient to confuse any reader) 65-128 : Slight encryption -- characters are inverted as before, but every few characters will be garbaged. 129-192 : Medium encryption -- more frequent garbage than 65-128. 193-255 : Total garbage. SHAREWARE_(THE_IMPORTANT_PART)__________________________________________________ My name is Richid Jernigan IV. In the past I have released several not- terribly-popular programs through PD, the most popular were developed for the //e'ers of the day (when AE's were so popular) by myself and the Masked Marauder... Super VH Mod I and ][. Anything with The Eye (old handle) or Huan-Ti plastered all over it is mine (I _am_ a bit vain. Had you noticed?). I have designed and written all the versions of MetalBase and all supporting code. I claim this code as mine alone, in all ways, shapes, and forms. HOWEVER, I learned most of what I know of programming through examples from various people, which were distributed freely. As such, I feel it's only right for me to try to contribute this, the only thing I can think of which anyone might want, to the public. I would therefore like to distribute this freely and without restriction. HOWEVER, I'm also broke. I've spent too long on this project to see it suddenly become part of everyone's wet dreams without any financial reward on my part. So, I'm releasing it as shareware. The conditions: * Give it to as many people as possible. * Leave the existing documentation intact. Add more if you'd like, but leave this. Feel free to add sample programs and relations if you like. * Leave the credits. I'm vain and I'm poor, and the publicity for me might do me good; if not for my wallet, at least for my ego. * Tell me about any bugs you find (God forbid -- you wouldn't _believe_ how many tests this thing has been through). * Send any modifications to me (as it sez in mbase.c, I'd like to see the improved code). More importantly, SEND IDEAS! If you'd like an improvement, tell me about it! Whether or not you modify the code yourself, I'd like to know what's being done to it. * If you find MetalBase useful, elegant (check out _search in mbase.c), or just plain worth it, and _certainly_ if you use it in any commercial or shareware programs, please send a donation ($5? $10? The more the better, really!) to the poor programmer's fund (hehehe): Richid Jernigan IV PO Box 827 Norris, TN 37828 <- Yes, folks, he's from TN! Gimme a name and a mailing address (internet or bitnet account names are fine -- just so I can distribute code / information), and you'll receive the following: * Any new versions of MetalBase, whenever they pop up * The privilege of being able to request modifications or development of any of the previously-mentioned utilities (which will be sent to everyone on the list when available) * The privilege of being able to DEMAND fixes to any bugs you find (now wouldn't _that_ be fun?) * A big smile from me (worth it, isn't it? Wreeetch!) I can also be reached (during the school-year, save for Spring '90-91) at (now) rpj@owlnet.rice.edu, or (soon) at richid@owlnet.rice.edu . Share and enjoy! (God, did I just say that? Yeech! I'm quoting books again.) ___ / / The man with the hammer, | (___________.----------------. Richid ( ___________| | (rpj@owlnet.rice.edu) ) ( `----------------' (Huan-Ti) '---`