Objectivity/DB Administration : Administration Tasks : Working With Distributed Systems
Working With Distributed Systems
You can distribute the files and processes of an Objectivity/DB system among the nodes of a network. This chapter provides information you should consider when setting up a distributed Objectivity/DB system, including:
General information about distributed Objectivity/DB systems.
Guidelines for setting up a distributed Objectivity/DB system.
Understanding Distributed Objectivity/DB Systems
An Objectivity/DB system consists of various processes and files:
The Objectivity/DB processes include your applications, as well as the lock server and data-server software such as AMS.
The Objectivity/DB files include boot files, data files (system-database, database, and container files), and journal files.
Certain of these files and processes are standard for every Objectivity/DB system—namely, the Objectivity/DB system resources for a federated database. These system resources include the boot file, system-database file, lock server, journal directory, and data-server software. See Objectivity/DB System for descriptions of these resources.
The other files and processes—that is, Objectivity/DB applications and the data files storing application-created data—may vary widely from system to system. For example, a small Objectivity/DB system may have one user application that accesses data centralized in a few database files. A large Objectivity/DB may have many user applications and custom utilities that access data in numerous database files and container files.
You can place the processes and files of an Objectivity/DB system all on the same computer, or you can distribute them among multiple machines on the same TCP/IP network. Within a particular system, some of these machines may be physically close (on a local-area network) and some may be physically distant from each other (on a high-bandwidth, low-latency wide-area network).
Figure 14‑1 Small Distributed Objectivity/DB System
Note:See the installation and configuration documentation on the Objectivity Developer Network for information about configuring TCP/IP on a Windows host in preparation for Objectivity/DB.
The physical location of data is transparent to an accessing application, so you can use Objectivity/DB tools to change the location of data files and system resources without having to recompile the application.
Objectivity/DB Hosts
An Objectivity/DB host is a network node (typically, a server, workstation or personal computer) that runs one or more Objectivity/DB processes or stores one or more Objectivity/DB files. The following terms refer to the hosts in a distributed Objectivity/DB system:
Client host
Example: Host1 in Figure 14‑1
Network node that runs an Objectivity/DB application and stores the boot file; sometimes called an application host.
Data-server host
Example: Host1 and Host 2 in Figure 14‑1
Network node that stores Objectivity/DB data files and journal files
Lock-server host
Example: Host3 in Figure 14‑1
Network node that runs an Objectivity/DB lock server. May also be a client or data host.
A particular client host can also be a data-server host in the same system. When this is the case, the Objectivity/DB files on the host are said to be local to the accessing application. Conversely, if a client host and a data-server host are separate network nodes, the application is said to access remote Objectivity/DB files. A lock server host may be a separate network node, or it may be the same as one of the client or data-server hosts.
A typical distributed Objectivity/DB system has:
Multiple data-server hosts to accommodate the various files. Database files are the primary units of distribution among these hosts; container files, if any, are normally stored near their parent database files.
Multiple client hosts to accommodate multiple Objectivity/DB applications.
One lock-server host.
Host Architectures
A heterogeneous distributed Objectivity/DB system includes hosts of different architectures, where a host’s architecture is a particular combination of hardware, operating system, and compiler. Objectivity/DB supports a variety of architectures on several platforms. See the Objectivity Developer Network for details about supported configurations.
An application is compiled for the architecture of its intended client host. Applications compiled for different client-host architectures are interoperable; for example, an application running on a Windows client host can access data created by an application running on a Linux client host, and vice versa. Each application can create and access files on a data-server host of any architecture, provided the host is running appropriate data-server software.
See Guidelines for Setting Up an Objectivity/DB System for information about accommodating hosts of different architectures. See Chapter 15, “Rehosting an Application,” for information about the impact of changing an application’s client-host architecture.
Advantages of Distribution
Distributing an Objectivity/DB system across multiple hosts enables you to:
Extend the system’s capacity for data storage and processing far beyond that of a single host.
Take advantage of a variety of computing and networking hardware.
Shorten the network path between an application and the data it accesses.
Accessing data across a network may add significantly to response time because of network overhead and contention. An application’s speed is likely to improve significantly if you store data files locally to your application.
Reduce the number of requests to be handled by any single network path, processor, or storage device.
Placing all data on the same data-server host can create a processing bottleneck—for example, due to competition by concurrent applications for files on the same physical drive. Distributing the files among different drives reduces such contention.
Provide some degree of fault tolerance. If one data-server host fails, applications may still be able to access data on other data-server hosts.
Consider a federated database that contains employee information for a chain of warehouse stores with three regional offices—in San Francisco, Chicago, and New York. Figure 14‑2 shows that a region’s employee information is distributed among several database files on different data-server hosts in the regional office.
Users at each regional office run applications that frequently access their local database files (across a local-area network) and occasionally access the database files of the other regions (across a wide-area network). Thus, an application running on a client host in the San Francisco regional office uses a local network to access the databases SFEmp1 and SFEmp2, but must occasionally go over a wide-area network to access the databases NYEmp1 and NYEmp2.
Figure 14‑2 Widely Distributed Databases in a Federated Database
Distributing the database files allows each database to be placed near the application that accesses it most frequently, as well as providing a degree of fault tolerance—for example, if the host containing SFEmp1 fails, users in the Chicago office can still access CHEmp1 and CHEmp2.
Guidelines for Setting Up an Objectivity/DB System
This section provides considerations and guidelines for configuring a heterogeneous distributed Objectivity/DB system.
Data-Server Hosts
You can locate Objectivity/DB data files and journal files on any combination of Windows and Linux hosts.
The way you share files determines how filenames should be specified to tools and applications that access the federated database and update its global catalog or any database’s catalog of containers. For information about the formats for specifying files on data-server hosts, see Specifying Remote and Local Files.
Serving Windows and Linux Client Hosts
You can run either of the following on a data-server host to make data files and journal files available to Windows and Linux client hosts:
Objectivity/DB’s Advanced Multithreaded Server (AMS).
Network File System (NFS).
AMS is recommended over NFS, because it improves update performance and it simplifies the specification of pathnames for distributed data files. For more information, see Deciding Whether to Use AMS.
Serving Windows Client Hosts Through NFS
An application that runs on a Windows client host and accesses an NFS data-server host has a user ID of 100. You must ensure that user 100 is defined on the Linux data-server host and has the necessary access permissions.
Windows-Only Environment
If you are using Windows data-server hosts, and the Objectivity/DB files on them will be accessed exclusively by applications on Windows client hosts, you can choose AMS or Microsoft Windows Network to make these files available. AMS is recommended over Windows Network because it simplifies the pathname specification for Objectivity/DB files.
If you choose to use Windows Network to share Objectivity/DB files residing on Windows data-server hosts, you must refer to these files using Universal Naming Convention (UNC) network share names. UNC names are of the form \\host\sharedDirectoryName\path. When you use UNC names in a tool or application, you must also specify the file host using the literal string oo_local_host.
Note:You may not designate remote or local Objectivity/DB files using mapped drive letters (such as Z: mapped to c:\project\myFD).
Client Hosts
You can run Objectivity/DB database applications on Windows or Linux client hosts.
Boot-File Location
Applications and lock servers use boot files to find the system resources of a federated database. In principle, only one boot file is necessary per federated database. However, in networks that contain multiple client hosts, it is sometimes convenient to maintain several copies of the boot file, one on each client host. This allows each application to use a local copy of the boot file.
If you maintain multiple copies of the boot file, it is strongly recommended that you specify one of these copies explicitly to the lock server to ensure that the lock server can resolve at least one boot-file pathname; see Setting Up Recovery in Mixed Environments. If you do not follow this recommendation, you must instead ensure that every copy has a pathname that can be resolved by the lock server.
Warning:If you use the ChangeFd tool to change federated-database properties, the changes are written only to the boot file you specify to the tool. It is your responsibility to copy the boot file and distribute the copies after such changes.
Setting Up Client Hosts for Remote Data Access Through NFS
When you use NFS to provide data access in a distributed Objectivity/DB system, you may need to adjust the data packet size on any Windows or Linux client host that will access files through NFS. In particular, you should check whether the TCP/IP protocol stack on the client host requires a smaller data packet size than 8192 bytes, which is the default used by Objectivity/DB with NFS. Furthermore, in a congested network, a Remote Procedure Call (RPC) timeout error message may also indicate that the data packet size is too large.
To adjust the data packet size:
Set the environment variable OO_NFS_MAX_DATA.
You do not need to adjust the data packet size if you use AMS instead of NFS.
Lock-Server Hosts
You can run the lock server on a Windows or Linux node.
When Objectivity/DB is distributed among network nodes of different architectures, you should follow the guidelines for locating the lock server in Setting Up Recovery in Mixed Environments. These guidelines enable the lock server to access Objectivity/DB files when performing automatic recovery.
Configuration Summary
The following table summarizes the possible combinations of platforms in a heterogeneous Objectivity/DB system.
Data-Server Hosts1
Catalog Entry Format
Allowed Client Hosts
Allowed Lock-Server Hosts2
Windows using AMS
host, c:\path
Windows using Windows Network
oo_local_host, \\host\path
Windows only
Windows only
Linux using AMS or NFS
host, /path
oo_local_host3, /path
1. AMS is recommended.
2. You should enable automatic recovery by starting the lock server with the boot file path as an argument. See Setting Up Recovery in Mixed Environments.