|
What is Aruna DB? |
Last Updated: 9/25/2001
A_Catalog.rb
This is a simple class to store and retrieve objects (key/value pairs) to/from a file. It is a cross between a PStore and a Hash. It is similar to PStore in that the objects are written to disk. It is different from a PStore in that objects are individually read from disk and written to disk. It is similar to a Hash in the since that key/value pairs are stored in the A_Catalog much like in a Hash. A Hash is actually used to store the keys. It is different from a Hash in that the value is written to disk and a pointer to the value (the file position) is stored in the Hash as the value to the key. A_Filestore, A_BTree, A_Table, and A_Index (see ArunaDB for more details) use this class to store information about persistent objects such as filestores, btrees, tables, and indexes. I recommend you suffix all catalogs with '.ctl' so they can be easily identified.
Stores and retrieves objects (key/value pairs) from a file. When you add an object it is written to the file immediately. When you access an object it must be read from the file. Marhsal.dump and Marshal.load are used to read and write the objects so you should be able to store most Ruby objects in A_Catalog. Objects are not cached in memory. Pointers to objects are cashed in memory so the objects are retrieved quickly. Each object is individually read and written to the file, as opposed to a Pstore, which reads and writes all objects at once. When you delete an object, it's position and size are remembered and automatically reused when possible. FYI, when you close an empty catalog, the file associated with the catalog is automatically deleted. This class is useful for storing small amounts of data that is not referenced or used very often. If you need to store a lot of data or you need frequent access to the data you should consider using a PStore or a A_BTree.
A_Catalog stores a lot of important information for ArunaDB. Make sure you back it up regularly and keep it safe. You should also back up all A_FileStore files that this catalog references at the same time. Losing a catalog or allowing a catalog to get out of sync with any A_FileStore file means losing all objects that the catalog points to. If you restore only one A_FileStore file and it has recently changed, the catalog and filestore could easily get out of sync and you could lose data. If you restore only the A_Catalog and a A_FileStore file has been recently updated, you could loss data. Always backup and restore your A_Catalog and A_FileStore files together as a single unit. Never restore one with the others.
This module is not thread safe. I have provided methods that allow you lock and unlock a catalog. If you are running multiple threads, you should lock catalog before accessing or updating the catalog and unlock it when you are finished. A synchronize method has been provided to make this easy and consistent with Mutex usage. Only one process can access the same catalog at the same time. Many threads can safely access the same catalog as long as you lock/unlock the catalog while you are accessing or updating it.
This module also adds methods a_conform and a_conform! to the Array and String classes. ArunaDB (A_FileStore, A_BTree, A_Table, and A_Index) always calls one of these methods against the object name (the key in the A_Catalog) before inserting the object in the system catalog. Having Tree1, TREE1, and tree1 be different objects is undesirable for most users so I use a_conform() to force all names (keys) to lower case before accessing the catalog. If you prefer the case sensitivity or you prefer all upper case names as Oracle does, then redefine these for methods to format the name (or key) to suit your needs. I put these new methods in with the A_Catalog class because these methods are only used on the key (or object name) when accessing the catalog. This provides me a consistent way to format the name (or key) of all objects in the catalog regardless of how users type in the names and it also allows me (or other callers) to easily change it. I choose not to imbed a_conform() directly into the A_Catalog class because this is how I choose to interact with the A_Catalog class in ArunaDB and I did not want to force other user's to conform to the same standard that I use. Feel free to use a_conform if it suits your needs.
cat = A_Catalog.use('tst_a_catalog.ctl')
cat['FileStore', 'fs1'] = 'filestore 1'
cat['FileStore', 'fs2'] = 'filestore 2'
cat['Index', 'table1'] = 'index'
cat['Index', 'table1.idx1'] = 'index 1'
cat['Index', 'table1.idx2'] = 'index 2'
cat['Table', 'tbl1'] = 'table 1'
cat['Table', 'tbl2'] = 'table 2'
print "All Items:\n"
cat.each {|type, name, value| print " #{type.ljust(12)} : #{name} = '#{value}'\n"}
cat.show
A_Catalog.show()
print "All Index.table1 items:\n"
cat.each_sorted('Index', 'table1') {|type, name, value| print " #{type.ljust(12)} : #{name} = '#{value}'\n"}
print "Deleting 'Index', 'table1.idx1'\n"
cat.delete('Index', 'table1.idx1')
cat.show
A_Catalog.close_all()
Close all open catalogs.
A_Catalog.connect(filename, pad=100)
Alias for use().
A_Catalog.drop(filename)
Drop the catalog and delete it's file.
A_Catalog.new(filename=nil, pad=100)
If the (filename) exists then open the catalog otherwise creates an empty catalog. See the description on A_Catalog.use() above for a description of pad.
- filename - is the fully qualified (including full path) filename of the catalog. If you don't supply a path your currently working directory is used. For example: "c:/my_dir/my_catalog.ctl". I use a .ctl extension on all A_Catalog files to make them easy to identify.
- pad - this pads each object as it is written to disk. This reduces the need to relocate changed objects and allowed free blocks to be resued more often.
A_Catalog.open(filename, pad=100)
Alias for use().
A_Catalog.show()
Show the default or system catalog, see the instance method with the same name for details.
A_Catalog.system_catalog()
Returns the catalog currently in use.
A_Catalog.version()
Returns the version.
A_Catalog.use(filename, pad=100)
Connect to and use a different default or catalog. Filename should include a full path. Only one process can be connected to a catalog at the same time. Multiple threads can successfully connect to the same catalog. Returns A_Catalog. If the filename exists, it is opened and used. If the filename does not exist, it is created. If you close and empty catalog, the filename is deleted.
- filename - is the fully qualified (including full path) filename of the catalog. If you don't supply a path your currently working directory is used. For example: "c:/my_dir/my_catalog.ctl". I use a .ctl extension on all A_Catalog files to make them easy to identify.
- pad - allows you specify how much to pad your objects as they are written to disk. Since Marshal.dump/load is used, the objects may change in size whenever you change and re-store an object. To reduce the possibility of have to relocate your object every time you change it, I allow you to pad the object. Pad is the minimum amount of space to write. For example if your object is 25 bytes, 100 bytes is actually written to disk because your object is padded to 100 bytes. If your object is then expanded to 125 bytes, your object will be relocated and 200 bytes will be written to disk. Set pad = 1 to eliminate all padding. This also reduces the number of free block in the catalog (because objects are not relocated very often) and makes re-using free blocks much more viable.
[](key)
Same as at(key).
[]=(key, value)
Same as insert().
at(key)
Returns the value previously stored for this key.
- key - can be any object that can be stored in hash, typically this is a string that indicates the name of the object you want to store. Typically, I use an array, such as [type, name], as the key where type is the type of object (A_BTree, A_FileStore, etc) and name is the name of the object.
clear()
Removes all key/value pairs from this catalog.
close()
Close the catalog. If the catalog is empty, the file is automatically deleted.
delete(key)
Delete this key from the catalog.
disconnect()
Alias for close()
each()
Iterate over each key/value pair.
each_key()
Iterate over each key in the catalog.
each_pair()
Same as each
each_root()
Same as each_key().
each_sorted()
Iterate over each key/value pair after the key is sorted.
empty?
Returns true if this catalog is empty, otherwise returns false.
exists?(key)
Returns true if this key exists in this catalog otherwise returns false.
export(filename)
Exports all items to the filename, be careful this will overwrite an existing file. Marshal.dump is used to write the objects to filename. This is useful for backing up your catalog. You can back up this file directly as long as no-one is updating it. This is used most often to eliminate unused space from the catalog.
fetch(key)
Same as at(key).
filename()
Returns the name (and path) of the file used by this catalog.
free_space()
Returns the number of unused bytes in the catalog.
has_key?(key)
Same as exists?(key).
import(filename)
Imports data from the filename, filename must have been previously exported. This is useful for removing empty space from your catalog, assuming you deleted some objects and the space has not been reused. Marshal.load is used to read the objects from filename.
include?(key)
Same as exists?(key).
insert(key, value)
Store the key/value pair in the catalog.
key?(key)
Same as exists?(key).
keys()
Returns an array of all roots that match the type/name criteria.
length()
Return the number of key/value pairs in the catalog
lock()
Works exactly like Mutex.lock. This locks the Mutex for this catalog.
name()
Returns the name (and path) of the file used by this catalog.
path()
Returns the name (and path) of the file used by this catalog.
root?(key)
Same as exists?(key).
roots()
Same as keys().
shift()
Same as Hash.shift
size()
Same as length.
show(key=nil, prefix='')
Show all objects in the catalog match key. If key=nil, then all objects are printed. Prefix will be added to the beginning of each key/value pair. This allows you to indent the key/value pairs as they are printed.
show_free_space()
Returns a nicely formatted sting showing the number of unused blocks and the amount of unused space (in bytes) in the catalog.
synchronize()
Works exactly like Mutex.synchronize. A Mutex is create for each catalog. This locks the Mutex for this catalog, executes the block, and releases the Mutex. For example:
cat = A_Catalog.use('test_catalog')
cat.synchronize do
perform your catalog updates here
end
unlock()
Works exactly like Mutex.unlock. This unlocks the Mutex for this catalog.
tst_a_catalog.rb - provides basic testing for this class.
9 - reading and writing to the system catalog
See a_debug for more details.