Objectivity/DB Administration : Administration Tasks : Federated Database Tasks
Federated Database Tasks
A federated database is the highest level in the Objectivity/DB storage hierarchy. It is the unit of administrative control for a “federation” of associated databases and contains the data model, or schema, that describes all classes of objects stored in these databases. Administering a federated database involves physical and logical restructuring, maintenance, and troubleshooting that affects the entire set of associated databases.
This chapter describes:
Background information about federated databases.
Getting information about a federated database.
Creating or changing a federated database.
Managing a federated database’s license.
Installing a federated database in a new operating environment.
Exporting and importing a federated database’s schema.
Managing the registry of storage locations for a federated database’s database files.
Deleting a federated database.
Troubleshooting access problems.
Getting transaction information.
Referencing objects in a federated database.
Estimating disk-space requirements.
About Federated Databases
Physically, a federated database consists of two files:
The boot file. This file contains the configuration information used to locate and open the federated database.
The system-database file. This file stores the schema for the federated database, a global catalog of the member databases and their storage locations, and the main storage group for registering the available storage locations for new databases.
Federated-Database Properties
A federated database has a number of properties. Two properties specify the federated database’s identity:
Integer identifier—the unique positive integer that identifies the federated database to the lock server. This number is chosen by Objectivity/DB.
System name—the unique name that identifies the federated database to Objectivity/DB. The system name is the same as the boot file name.
The following properties specify the locations of the Objectivity/DB system resources associated with the federated database:
System-database file host and file path—the host system and full directory path (including the filename) of the federated database’s system-database file.
Journal-directory host and path—the host system and directory path for the journal files, which store information for Objectivity/DB to use when rolling back incomplete transactions for all member databases.
Lock-server host—the host system on which the lock server for the federated database is running.
Boot-file host and path—the host system and directory path for the boot file of the federated database.
Other properties specify:
The storage-page size (in bytes) of the system database. See Pages and the Objectivity/DB Cache.
The schema policy of the federated database—whether schema updates are allowed. See Federated-Database Schema Policy.
The values for these properties are set when a federated database is created. You can modify various properties of an existing federated database using Objectivity/DB administration tools; see Changing Federated-Database Properties.
Federated-Database Licensing
Every federated database you create maintains a persistent object that represents your license for using Objectivity products. The license object in each federated database contains encrypted data summarizing the terms and conditions of your licensing agreement—for example, the Objectivity products you are authorized to use, the usernames of any other authorized users, the architecture(s) on which development will take place, and so on.
Whenever a tool or database application attempts to open a federated database, the federated database’s license is checked to verify that such access is permitted. If access is denied—for example, because the license has expired or is invalid for the Objectivity product you are attempting to use—the tool or application will report an error and terminate.
You initialize a federated database with your current license when the federated database is created. You must update the federated database’s license whenever you receive new a license from Objectivity—for example, after purchasing a new product or licensing new users. A new license is typically delivered to you electronically as an encrypted text file. See Managing a Federated Database’s License.
Federated-Database Schema Policy
A federated database’s schema policy determines whether its schema is read/write or read-only. The schema of a newly created federated database is read/write, so descriptions of application-specific classes can be added to it. As long as the schema is read/write, it can be modified in various ways—for example, by:
Adding class descriptions with ImportSchema.
Performing schema-evolution operations using the mechanism provided by your Objectivity/DB application programming interface.
Using a Spark driver application to ingest an Apache Spark DataFrame.
At any time, a schema policy can be changed to read-only, so that subsequent attempts to modify the schema will fail with an error. For example, you could set a federated database’s schema policy to read-only during development, to prevent unintended modifications from being made.
The schema policy can be changed back to read/write whenever necessary to permit schema evolution. See Changing a Federated Database’s Schema Policy.
Getting Federated-Database Information
You can get information about a federated database, including the current values of its properties. Such values are typically used as input to other tools.
Listing All Component Files
To list the component files of a federated database, run DumpCatalog with the -bootFile option.
The output includes the full pathnames of the following items:
The system-database file.
The boot file.
The journal directory.
All database files.
All container files.
The output also includes identifiers, system names, and other properties of each component.
Listing Properties
To list the current values of a federated database’s properties:
Run ChangeFd with the -bootFile option.
As an alternative, you can inspect the boot file to view the current values for most properties. However, you must never edit this file directly.
Determining File Type
To determine the type of a file belonging to a federated database, run FileInfo with the name of the file. The output indicates whether the file is a boot file, database file, and so on. By convention, the type of a particular file is indicated by its filename extension, although specific extensions are not required; see Filenames.
Comparison of Tools That Display Properties
Table 3‑1 compares the federated-database properties that are displayed by ChangeFd and FileInfo.
Table 3‑1: Properties Displayed By Tools
System name
System-database file host and path
Boot-file host and path
Journal-directory host and path
Lock-server host
Storage-page size
Schema policy
Internal database format
Creating a Federated Database
Planning for the New Federated Database
Before you create a federated database:
1. Decide whether the new federated database will be distributed among multiple hosts. If so, you should identify:
The data-server hosts, client hosts, and lock-server host.
The host architecture(s).
The data-server software will you run on each data-server host.
See Distributed Objectivity/DB Systems and Chapter 13, “Working With Distributed Systems.”
2. Identify the specific locations of the system-database file, journal directory, and boot file of the new federated database. These locations can be on the same data-server host or on different data-server hosts.
For best performance, the journal directory should not contain the system-database file and should not be the journal directory of any other federated database.
3. Determine the format you should use to specify file and directory paths when you create the federated database. The format for a file path depends on the data-server host’s architecture and data-server software. For details, see Filenames and Host and Path Formats.
4. Choose the federated database’s system name. The system name will be the simple name of the boot file.
Procedure for Creating the New Federated Database
Note:If you are creating a federated database for use with a Spark driver application running in a cluster, consider using the shortcuts in Setting Up Storage on a Cluster instead of performing the steps in this section.
To create a federated database:
1. Verify that a lock server is running on the intended lock-server host for the new federated database. If necessary, start it; see Starting a Lock Server.
2. Verify that your license file is set up. If necessary, set it up; see Setting Up a License File.
3. Run CreateFd.
Specify the -fdName option.
Specify additional options to set nondefault values for the other properties. Include the -licenseFile option if your license file is in a nondefault location.
4. Arrange for class descriptions to be added to the schema of the newly created federated database, if required by your Objectivity product. See Mechanisms for Adding Class Descriptions to a Schema.
UNIX, Linux, and Macintosh Examples
Simple Federated Database
Assume you want to create a federated database called projectFD in the /project1/data directory on a UNIX, Linux, or Macintosh data-server host. That is, you want the chosen directory to:
Contain the new boot file (projectFD.boot) and system-database file (projectFD.fdb).
Serve as the journal directory for the new federated database.
Assume you want the current host to be the lock-server host.
Example After starting the lock server on the current host, you change to the chosen directory and enter the CreateFd command.
cd /project1/data
objy CreateFd -fdName projectFD
Distributed Federated Database
Assume you want to create a federated database called projectFD in the /project1/data directory on a remote UNIX, Linux, or Macintosh data-server host called machine55. Assume further that:
The lock-server host is to be machine95.
The journal directory is to be created in its own directory on machine55.
The host machine55 is running AMS as its data-server software, because the new federated database is to be accessed by applications and tools running on a mixture of client hosts.
Example After starting the lock server on machine95, you enter the CreateFd command as shown.
Because machine55 is running AMS, you must specify both the hostname and fully qualified pathnames that are local to that host (see Files on AMS Data-Server Hosts).
objy CreateFd -fdName projectFD fdDirHost machine55 
	-fdDirPath /project1/data -lockServerHost machine95
	-jnlDirHost machine55 -jnlDirPath /project1/journalfiles
The same command could be used if NFS is used as the data-server software, provided that /project1 is the name of an NFS-exported file system.
Changing Federated-Database Properties
You can change various properties of a federated database, usually to accommodate system or network changes or to improve application performance. The following properties can be changed:
Host and path of any Objectivity/DB system resource.
Schema policy.
Note:You cannot change a federated database’s system name or the storage-page size of its system database.
A federated database must be fully operational before you can change its properties. If necessary, install the federated database first; see Installing a Federated Database.
Moving a Federated Database’s System Resources
You can move one or more of a federated database’s Objectivity/DB system resources by changing the corresponding host or path properties.
To change a host or path property of a federated database, run ChangeFd with the -bootFile option and the appropriate options from the Table 3‑2.
Table 3‑2: Options for Changing a Federated Database 
To Change This Resource’s Host or Path
Use These ChangeFd Options
System-database file
Boot file
Journal directory
Lock-server host
(See Changing Lock-Server Hosts for the complete procedure.)
Example The following command moves the system-database file myFd.FDB to another data-server host (mach77) and directory (/data2):
objy ChangeFd -sysFileHost mach77 -sysFilePath /data2/myFd.FDB -bootFile myFd
The change operation updates the specified location properties in the boot file and global catalog. In addition:
Changing the properties of the boot file creates a new copy of the file in the file system. You delete the old boot file using an operating-system command.
Changing the properties of the system-database file physically moves (or renames) the file in the file system. Note: If you do not want Objectivity/DB to move the file, you can add the ‑catalogOnly option, and use an operating-system command to physically move or rename the file.
Changing a Federated Database’s Schema Policy
To change a federated database’s schema policy to be read-only or read/write:
Run ChangeFd with the -readOnlySchema or -readWriteSchema option, and the -bootFile option.
Managing a Federated Database’s License
Every federated database is initialized at creation with an Objectivity license that authorizes access to it. Whenever your Objectivity license changes—for example, because you have purchased new Objectivity products or upgraded your current products to a new major release—you must update the license in each federated database that will be accessed under the new terms.
This section describes various tasks for managing licenses:
Setting Up a License File.
Updating a Federated Database’s License
Verifying a Federated Database’s License
For information about licenses, see Federated-Database Licensing.
Setting Up a License File
A license file is a text file containing an encrypted Objectivity license. You set up a license file each time you receive a new encrypted license. Setting up a license file makes your license available to the Objectivity/DB tools that initialize or update federated databases. To set up a license file for a new Objectivity license:
1. Create a new default license file:
a. Check whether a file called oolicense.txt already exists in your ThingSpan installation directory installDir. If so, rename that file.
Typical ThingSpan installDir locations are as follows:
C:\Program Files\ThingSpan\version
UNIX (Linux)
b. Save your new Objectivity license as a text file called oolicense.txt in your ThingSpan installation directory installDir.
Note: If you received your encrypted license by electronic mail, you can save the attached file from your mail reader, or you can create an empty text file and paste the license text into it.
2. (UNIX, Linux, and Macintosh only) If you have a personal Objectivity license that is different from the one shared by your entire installation, create a new personal default license file:
a. Check whether your home directory already contains a file called oolicense.txt; if so, move or replace that file.
b. Save your new personal Objectivity license to a text file oolicense.txt in your home directory.
3. Make a backup copy of the new license file in another directory for safekeeping. A useful convention is to include the license’s identifier in the filename. (The identifier is included in the electronic mail message or other file accompanying the encrypted license.)
The default license files created in steps 1 and 2 enable you to create new federated databases or update their licenses without explicitly specifying a license file. (On UNIX, Linux, or Macintosh, a personal default license file overrides any general default license file in installDir.) If your site conventions require it, you can choose a nondefault name and location for your license file; you must then specify the license file explicitly whenever you create a federated database.
Note:If you are authorized to use multiple Objectivity products, all authorization is combined in a single license. Thus, you need to set up only one license file for your current combination of products (rather than setting up a separate license file for each product).
Once a federated database is initialized or updated from a particular license file, the file can be moved or renamed without affecting the license in the federated database. For convenience, the current license file can be copied to multiple locations.
Updating a Federated Database’s License
Whenever you set up a new license file containing a new encrypted license, you must explicitly update each federated database that will be accessed under the terms of the new license.
To update a federated database’s license, run License with either the -licenseFile or -fromDefault option, and with the -bootFile option.
Updating a license replaces the federated database’s previous license with the new one. Your old license file is your only backup.
Verifying a Federated Database’s License
Occasionally, you may need to verify the validity of a particular federated database’s license—for example, to make sure the federated database has the most up-to-date license.
To display a federated database’s license, run License with the -bootFile option and no other options.
The license is displayed in plain (unencrypted) text. You can check whether the displayed license’s identifier matches that of your latest license.
Installing a Federated Database
Installing a federated database sets it up after its files have been placed in a new operating environment.
Note:This task applies only to a federated database whose data files are all located within a single directory. You typically perform this task while setting up for Objectivity/DB tutorials.
The new operating environment may be a host in the original network or in a different network. Consequently, the relocated federated database may need a different lock-server host, as well as different host and path properties for its data files and journal directory. However, the network or operating-system commands used for transferring a federated database’s files do not change any of the properties stored in those files. Installing the federated database enables you to adjust these properties so they refer to the hosts and paths of the new operating environment.
Note:Installing a federated database differs from changing its properties (see page 40), because Objectivity/DB change operations can be performed only on a federated database whose properties and catalogs are already consistent with the current operating environment.
Basic Procedure
Note:Due to a current limitation, this tool installs a federated database only if its database files were originally created in a single directory.
You can use the basic procedure for installing a federated database in a new operating environment, if all the application-created data files fit into a single directory:
1. Put the system-database file and boot file in the desired location(s) in the new environment.
2. Put all application-created data files (that is, all database files and any container files) in a single directory in the new environment.
3. Verify that a lock server is running on the desired lock-server host in the new environment, and start it, if necessary; see Starting a Lock Server.
4. Run InstallFd with the -bootFile option. Depending on where you put the system-database file, the data files, and the lock server, you may also need to specify the -fdFilePath, -dbDirPath, and -lockServerHost options. You can also include an option to change the federated database’s Objectivity license.
Exporting and Importing a Federated Database’s Schema
You can export the contents of a federated database’s schema to a textual XML document and then import the XML file into another new or existing federated database. To do so, use the ExportSchema and ImportSchema tools.
Note:For more information, see Exporting and Importing a Schema.
Managing the Registry of Storage Locations
A federated database’s main storage group (MSG) is a registry of storage locations in which new database files can be placed; see Main Storage Group and Global Catalog.
Following is an overview of registry-management tasks:
Register new storage locations in the MSG, and optionally define zones consisting of related storage locations; use AddStorageLocation.
Display the current contents of the MSG; use ListStorage.
Remove a storage location from the MSG; use RemoveStorageLocation.
Specify policies that govern:
The order in which registered locations are selected when placing new database files.
The preferred response if registered locations are found to be unavailable.
See Chapter 4, “Specifying File Storage,” for complete details about how to perform these tasks, as well as tasks that prioritize some storage locations over others for particular applications.
Note:In a federated database to be used with a Spark driver application, most registry-management tasks are set up with a script; see Setting Up Storage on a Cluster.
Deleting a Federated Database
You can delete an entire federated database, including all its files. To do so, run the DeleteFd tool with the -bootFile option.
Troubleshooting Federated-Database Problems
Checking Access Problems
If an Objectivity/DB tool or application cannot access a federated database, first verify that these conditions are true.
The federated database boot file is readable at the pathname specified.
A lock server is running on the specified host system and is compatible with the system-database file (unless the tool is running in standalone mode, which does not require a lock server).
The system-database file specified in the boot file is visible on the network.
The journal directory is visible on the network.
Appropriate access permissions (page 27) are set.
The federated database’s license (page 43) is valid for the accessing tool or application.
If these conditions are all true, it is likely that a tool or application failed to release a lock on the federated database. To locate unexpected locks, run LockMonitor. To release them and roll back the transactions that started them, run CleanupFd. For more about cleanup and recovery, see also Performing Manual Recovery.
Checking Consistency Problems
You can check the consistency of an entire federated database or a portion of it. To do so, visit the Objectivity Developer Network to obtain the oocheck tool.
You can use oocheck to check:
The entire federated database. This includes the entire storage hierarchy, including all persistent objects and their relationships (associations), on all logical pages, in all containers, in all databases.
A specific portion of the federated database’s storage hierarchy. You can check a particular database down to a specified level of detail—for example, to the containers in the database or to the objects in them.
The federated database’s schema
A particular journal file.
You can save the generated report to a file.
Fixing Consistency Problems
If oocheck reports errors, you should call Objectivity Customer Support to obtain help fixing these errors. In some cases, you may be directed to apply fixes by running the interactive tool oofix.
Warning:Unless you are very experienced with Objectivity/DB, you should not run oofix without assistance. Always make a backup copy of the affected database files before applying any fixes with oofix.
Getting Transaction Information
You can list all active transactions that have locks on data in a federated database. To do so, run CleanupFd with only the -bootFile option.
You can use ListWait with the -bootFile option and various options to list transactions that are waiting on any lockable Objectivity/DB object (federated database, database, or container). This tool also finds out whether a specified transaction is waiting for a lockable object, and if so, which transactions currently hold the lock on that object. Table 3‑3 summarizes the options for filtering ListWait information.
Table 3‑3: Listing Transactions Using oolistwait
To See
Use This Option
All waiting transactions
(no options)
Waiting transactions started on a specific host system
Waiting transactions started by a specific user
Waiting transactions started by a specific user on a specific host system
‑host and ‑user
A specific transaction’s status, and any transactions that are using resources that it needs
The ListWait tool lists transactions in the order in which they are retrieved, not in the order in which they will run (that is, not in a sequenced queue).
Referencing Objects in a Federated Database
Every object in a federated database can be referenced through a unique identifier that distinguishes it from other objects of the same type. A federated database’s identifier (sometimes called a reference number) distinguishes it from the other federated databases using the same lock server; a database’s identifier distinguishes it from the other databases in the same federated database; a container’s identifier distinguishes it from the other containers in the same database; and so on.
For most types of Objectivity/DB objects, the identifier is a single integer that serves as a key for locating each object relative to the storage object that contains it in the logical storage hierarchy. For example, within a federated database’s global catalog, each database identifier is mapped to a particular database file; within a database’s catalog of containers, each container identifier is mapped to a particular set of storage pages either embedded in the database file or stored externally in a separate container file.
The identifier of a persistent object, called an object identifier or OID, contains enough information to distinguish it from every other persistent object within the entire federated database, not just within the object’s container. An object identifier is 64 bits long and is composed of four 16‑bit fields in the following string format:
Database identifier.
Container identifier.
Logical page number in the container. A logical page is a storage page containing one or more small objects or the header information for a large object.
Slot number on the page. A slot is the portion of a logical page in which a single small object is stored. The slot number for a large object is always 0.
For example, 78-112-8-3 identifies the persistent object stored in slot 3 of page 8 in container 112 of the database whose identifier is 78. Objectivity/DB identifies objects using object identifiers instead of memory addresses because object identifiers provide interoperability across architectures, access to more objects than direct memory addresses permit, and runtime access to objects located anywhere in a network.
For uniformity, you can specify storage objects using either an integer identifier or a corresponding object identifier in the D‑C‑P‑S format:
A database identifier D corresponds to an object identifier D‑0‑0‑0. For example, the database with the identifier 78 can be specified as 78-0-0-0.
A container identifier C within a database D typically corresponds to an object identifier DC‑1-1, although the page and slot number are not 1 in all cases; see Container Identifier Formats. For example, 78-5-1-1 identifies the container 5 in the database 78.
Note:The identifier of a federated database should always be specified as an integer, not a four-part object identifier.
Object identifiers are reported in some errors to identify particular objects. If a persistent object is moved, its object identifier changes; if a persistent object is moved or deleted, its (old) object identifier will be reused for any new persistent object subsequently created in the same location. Similarly, the integer identifiers of previously deleted databases or containers will be reused as needed.
Estimating Disk-Space Requirements
The following subsections describe how to estimate the initial and maximum disk-space requirements for an Objectivity/DB federated database.
Estimating Initial Requirements
Objectivity/DB requires a certain amount of overhead disk space in addition to the disk space required by actual objects. Use the following formulas to estimate the initial amount of disk space required (before objects are created):
diskSpace  fdOverhead + (numberOfDbs * dbOverhead)
Approximately 100 kilobytes to 1 megabyte.
Number of databases in the federated database.
dbOverhead  numberOfConts * initPages 
Number of containers in a database.
Initial number of pages allocated for a container (see the language-specific documentation for information about creating containers).
Estimating Maximum Federated-Database Size
You can use the information in the following subsections to estimate the maximum size (in bytes) of a fully populated federated database and the approximate number of accessible objects it contains. The subsections describe:
Values used in size estimates, below.
Formulas for estimating the maximum size of a container (page 52), database (page 52), and federated database (page 52).
Formula for estimating the maximum number of objects (page 53).
Values Used in Size Estimates
Table 3‑4 lists various values you will need for estimating federated-database size. Several of these values—the maximum number of databases, containers, and pages—are determined by the field sizes in an object identifier. Each 16-bit field can have up to 216 values. This field size determines the addressing capability at each level of the storage hierarchy; some of these addresses are reserved for Objectivity/DB to use.
Table 3‑4: Values Required for Calculating Federated Database Size 
Maximum number of databases in a federated database
216 – 2 databases (65,534 databases)
Includes the system database and up to 65,533 application-created databases.
N databases contain internal data used by the placement system (one for each container placer with its own scope), and up to 65,533 – N databases may contain application-created persistent objects.
Maximum number of containers in a database
216 – 1 containers (65,535 containers)
Includes containers for Objectivity/DB housekeeping data as well as containers for application-created data.
The maximum number of containers in a database does not distinguish between embedded and external containers. In practice, however, the number of embedded containers is limited by the maximum database-file size (in bytes), divided by the average size of an embedded container in that database:
(db file size) / (average pages per embedded cont x db page size)
Maximum number of logical pages in a container
216 – 2 logical pages (65,534 logical pages)
A logical page is a storage page containing one or more small objects or the header information for a large object.
Minimum number of storage pages in a container
2 pages per container
Includes 1 page reserved for Objectivity/DB.
Maximum storage-page size
216 bytes (65,536 bytes)
Minimum storage-page size
210 (1024 bytes)
Overhead per object
14 bytes on a page owned by a 32-bit architecture
22 bytes on a page owned by a 64-bit architecture
The owning architecture of a page determines the disk format of objects on the page, and is set by the application that created the page. By default, an application uses the architecture of the client host on which it is running.
Average number of objects (slots) per logical page
(Page size - 32) / (Average object size + Overhead per object)
Although the theoretical maximum number of slots on a page is 216–1, Objectivity/DB reserves 32 bytes per page for overhead, in addition to overhead for each object (see preceding cell in this table).
The available space on a page for objects is therefore page size - 32. The average slot size is the average size of an object plus its overhead. The larger the objects, the fewer slots per page.
Formula for Estimating Maximum Container Size
The following formula calculates the approximate maximum number of bytes required by a container, based on the addressing capability of object identifiers. This formula uses the maximum storage-page size (216 bytes) and applies to embedded or external containers.
Maximum number of bytes in a container 
216  bytes/page  x  216  pages/cont
232 (or approximately 4 x 109)  bytes/cont
This formula takes into account only the logical pages in a container (the pages storing small objects and the header information for large objects). Containers that store large objects could be significantly larger due to the additional storage pages for large-object data.
Formula for Estimating Maximum Database Size
The following formula calculates the approximate maximum number of bytes required by a database, based on the addressing capability of object identifiers. This formula rounds the maximum number of containers per database to 216 and uses the estimated container size given in the previous section.
Maximum number of bytes in a database 
232  bytes/cont  x  216  conts/db
248 (or approximately 3 x 1014)  bytes/db
For databases containing only embedded containers, the estimate is the theoretical maximum size of a single database file. (The practical capacity of such a database is usually smaller, however, to keep the database file size within the limitations of file-management tools or physical media capacity.)
For databases with external containers, this estimate is the total amount of space needed by the database file and the related container files.
Formula for Estimating Maximum Federation Size
The following formula calculates the approximate maximum number of bytes required by a federated database, based on the addressing capability of object identifiers. This formula rounds the maximum number of databases to 216 and uses the estimated database size given in the previous section.
Maximum number of bytes in a federated database  
248  bytes/db  x  216  dbs/fd
264  (or approximately 2  x  1019 bytes/fd
Formula for Estimating Number of Objects
The following formula expresses the approximate number of accessible objects in a fully populated federated database. In this estimate, N represents the average number of objects per logical page, which is calculated as shown in Table 3‑4.
Number of objects 
N objects/page   x  216 pages/cont  x  216 conts/db  x  216 dbs/fd 
(248  x  N) objects/fd
This formula does not take into account the space used for container overhead, which varies depending on how full the container is.