Section 3: Software Architecture

Section 1 explained our system architecture, which included:

Both kinds of DAACs will be built from the same kind of hardware components (commodity processors, disks, tapes, and networks). Both kinds will run the same base software: UNIX, C, Fortran, and SQL-*. This section discusses the software architecture of these DAACs. In particular, it discusses

EOSDIS will have a huge collection of archival data spread among 2+N sites. Describing, managing, and selectively retrieving this data are central to the DAAC mission. Consequently, we have taken a database-centric approach to managing the DAAC dataóthe only way to automate the many tasks that the DAACs must perform. Automation will make the data more accessible to clients and will reduce costs for support and operations staff sharply.

In approaching the EOSDIS design problem, we tried to buy commercial off-the-shelf (COTS) software whenever possible. We define COTS as products with $10M/year revenue streams. Using such products has two advantages: They will evolve to protect the investments of the community, and they will allow cost-sharing among the large user community. In some areas it is easy to recognize standardsócurrent COTS operating systems (MS/Windows and UNIX), file systems (NFS), programming languages (C and Fortran), and network software (TCP/IP and SNMP) seem adequate for EOSDISÍ needs without much innovation.

The demands of EOSDIS are beyond the current state of the art in other areas because EOSDIS is much larger than any current system. In most of these areas, COTS products are moving in the right direction. In certain areas (for example, object-relational SQL-* DBMS engines), COTS products will meet EOSDIS needs in the required time frame. As a result, no special action is required. In other areas (for example, SQL-* middleware), COTS products will not meet the EOSDIS timeline. In these cases, we recommend that NASA contract with COTS vendors to accelerate their development timelines.

In two areas EOSDIS needs are unlikely to be met by COTS products. One is unique to EOSDIS: A type library for geo-mapped data. The second is unique to the architecture we have chosen: A scheme for managing the work flow in Figure 1-2. As a result, HAIS should initiate software efforts in these areas.

In summary, EOSDIS should

We now turn to detailing the elements in our architecture.

3.1 SQL-*

Phase 0 of EOSDIS took a conservative approach, storing all EOSDIS data in files and using SQL to store the EOSDIS metadata describing the files and the data lineage. This is a good first step, but it still relies on the file system directory name space to track and manage files. For example, each AVHRR image is a file with a name like:

/SuperDAAC/AVHRR/Lat38_lon44/date_1994_08_29/time_1130

Under the Phase 0 design, each superDAAC would have one hundred million such files by the year 2005. The directory for such a file space becomes a database. But it lacks a query language to search for data and a data management system that allows flexible data tiling, data joining, or parallel data search.

We propose to store all the data inside the database system and tile it within a spatial access method. The database will be stored within a file system, but typically there will be a few files per physical disk. The data management system will cooperate with the hierarchical storage manager to move data tiles between the nearline tape archive and the disk pool. This approach gives many more opportunities to optimize data access and find and exploit parallelism. It is also the only way we know to automate the placement of data, thereby providing location transparency. In addition, SQL is becoming a standard interface between clients and servers. Many graphical interface tools (GUIs) and high-level languages (4GLs) appearing on the desktop use SQL as their interface to database and transaction servers.

Today the $10B/year database industry is building from a base of the SQL-2 standard defined in 1992. Vendors are adding object-oriented extensions, and the SQL standard is evolving to include these extensions. The details are still being debated, but the outlines of the new SQL standard are fairly clear.

This report uses the term SQL-* to describe the language that will result from this evolution. From an EOSDIS perspective, SQL-* adds the following key object-oriented constructs to SQL-2:

These capabilities are best explained by examples abstracted from the Project Sequoia 2000 Benchmark. A subset of the schema for this benchmark is
-- a gridded AVHRR raster image in Lambert-azimuthal projection table field typecreate table RASTER ( time Time, -- time image was taken location Box, -- bounding box of image band Spectrum, -- spectral band data Array2D -- array of points );-- a USGS file describing land uses of various areas
create table POLYGON ( landuse Landuse, -- coded land use location Polygon -- the edges of the area );

Here, the RASTER table stores gridded (Lambert-Azimuthal) AVHRR satellite images collected over a period of time. The image has been gridded and tiled into USGS quadrangles. Hence, the AVHRR image can be identified by the date on which it was measured, the spectral frequency of the measurement (band), and the geographic rectangle of the image (location). The second table classifies geographic regions as polygons with a specific land use.

Notice that arrays, boxes, and polygons are required as data types to support this schema. SQL has no such data types. Object-oriented extensions to SQL allow application designers to extend SQL in this way.

Some example queries from the Project Sequoia 2000 benchmark illustrate the need for user-defined functions. The statement to request AVHRR imagery for a specific time, band, and rectangle would be:
select clip (location, RECTANGLE, data)from RASTERwhere band = BAND and time = TIME and overlaps(location, RECTANGLE)

Here, the three uppercase constants are run-time parameters that denote, respectively, a rectangle corresponding to the desired geographic area of interest, the wavelength band required, and the time required. To perform this query, the system must have 2 user-defined functions, overlaps() and clip(), to subset the data according to the userís desired study area.

To execute this query efficiently, the local DBMS must have a spatial access method, such as quad trees, grid files, or R-trees to limit the search to AVHRR images that overlap the rectangle. Otherwise, the query will require a sequential search of the entire database for images for that date and band. The best way to support this functionality is to allow user-defined access methods so that, if the local DBMS does not come with an appropriate access method, problem-domain experts can implement one.

If one drops the time restriction, the query becomes a search of all AVHRR images of a particular area and spectral band over time:
select clip (location, RECTANGLE, data)from RASTERwhere band = BAND and overlaps(location, RECTANGLE)

By the year 2005, this will be a petabyte search. For small areas, spatial indices can reduce this request to the retrieval of few thousand images. Since most EOSDIS data will be on tape tertiary memory, if images are retrieved one at a time this query will take several days. (In Section 3.4 we discuss middleware solutions that will speed up such queries through parallelism or lower their cost by using batch processing.)

This example shows the benefits of SQL. The non-procedural access of SQL dramatically reduces the programming effort needed to access EOSDIS data. Programming such a query using a file system would be labor-intensive, requiring the extraction of the rectangle and spectral band from the 3D data set and would probably not produce a parallel data retrieval scheme. It might be done with NetCDF or HDF, but it would be a lot of work.

Any query that combines data from 2 or more data sets would require much more programming effort. Consider the simple case of requesting raster data for a given land use type in a study rectangle for a given wavelength band and time. It is stated in SQL as follows:
select POLYGON.location, clip(RASTER.location,POLYGON.location, data) from RASTER, POLYGON where POLYGON.landuse = LANDUSE and RASTER.band = BAND and RASTER.time = TIME and overlaps(RASTER.location, POLYGON.location)

Here, LANDUSE gives the desired land use classification, and BAND and TIME specify the wavelength band and time of the desired raster data. The join between RASTER and POLYGON is declared by the overlaps() function. Finding this data using Fortran, an NFS file system, and the HDF libraries would require at least 1 week of programming.

Spatial data access, spatial subsetting, and array processing are all required to perform the above examples. Without them, the queries become extremely difficult (or impossible) to perform and require substantial application programming and non-COTS software.

SQL-* systems with these capabilities are available today. EOSDIS can prototype with 1 or more of these systems, confident that a wide variety of choices will appear over time. Currently, HP, Illustra and UniSQL support query languages with types, functions, and access path declaration. Oracle, IBMÍs DB2/2, and Sybase promise these features within the year. Many object-oriented DBMSes have substantial support for SQL in their products. They can be expected to have capable database engines in 2-3 years. Hence, the object-oriented extension mechanisms for SQL and COTS SQL engines will certainly be mature in the EOSDIS time frame (1997).

Table 3-1 summarizes the recommendations of this section.

Table 3-1: Recommended Actions and Costs
Recommended action
Cost ($)
Pick 1 or more COTS SQL-* system to prototype with.
0

3.2 Type Extension and Database Design

SQL-* allows type extension, but it does not necessarily define type libraries. Type libraries for text, images, spreadsheets, drawings, video, and other desktop objects will come from Microsoft, Apple, Adobe, Lotus, and other leaders in those areas. Standard libraries for scientific data have emerged in the form of NetCDF and HDF.

NetCDF placed particular emphasis on the ARRAY data type. It provides a rich set of functions to edit and extract arbitrary sub-rectangles (hyper-slabs) of an array. It stores arrays in platform-independent formats (XDR) and presents results in either Fortran or C array formats. NetCDF can store multiple arrays in a file and has a metadata or catalog area that can describe the dimensions, units, and lineage of the array.

HDF, which now subsumes NetCDF, is more ambitious. It has a hierarchical directory structure of objects within a file. Each object has a type and some metadata. HDF supports the array type with several type libraries (including NetCDF). In addition, it supports 3 kinds of color pallets, a text type, and a tabular data type similar to SQL. The text and table libraries lack the rich retrieval operators of a text-content-based retrieval or SQL relational operators. Rather, they are more like record-oriented file systems.

In addition to these programmatic type libraries, both NetCDF and HDF come with browsers that allow UNIX, MS/Windows, and Macintosh users to view and navigate a data set.

The success of NetCDF and HDF demonstrate the usefulness of type libraries. But neither NetCDF nor HDF will scale to a large database. They work within a single file (typically less than 1 GB), provide no data sharing, and provide very primitive navigation through the data (just the hierarchical name space). We expect the NetCDF ARRAY class library to be an important type in the EOSDIS type library and that the text types of HDF will be replaced by a richer document type, perhaps Microsoft's OLE type or Appleís OpenDOC. We expect the HDF tabular type will be replaced by SQL tables. The NetCDF and HDF metadata would move into the EOSDIS schema.

In contrast, we recommend that HAIS take the lead in defining an Earth Science type library appropriate to SQL-*. Several standardization efforts are already underway, notably the work of SAIF, OGIS, SQL/MultiMedia (SQL/MM), and POSC. The early experience from Project Sequoia 2000 is that these efforts are good starting points but require substantial extension to be useful to EOSDIS. NASA and HAIS should make a major effort to influence these activities so they become even more useful over time. They should allocate 2 people for the next 5 years to this task.

In addition, EOSDIS requires the following efforts:

Collectively, the type libraries, schema, and data dictionary define an EOSDIS-specific dialect of SQL-*. This dialect will evolve over time and whenever possible should converge with type libraries from standards organizations. Moreover, whenever possible, it should use public-domain type libraries to keep the cost low.

The EOSDIS effort should expect to have a substantial group of database administrators and type-designers performing these tasks. Moreover, EOSDIS will need a mechanism whereby users can define and register their own types, functions, and dictionary elements, thereby enhancing the base schema. In this way, the user community can participate in the evolution of the EOSDIS schema.

The Phase 0 EOSDIS and Project Sequoia 2000 experiences in this area are sobering. They found database design and type extension to be very difficult and time-consuming. Not only are EOS data complex and challenging to model, but there are many points of view from which this modeling can begin.

As a result, a team of 5-10 people, led by Earth scientists and including programmers and database designers, will be needed to define the schema as a multi-year evolutionary effort. EOSDIS should put the best people on this activity to enhance the chances of long-term evolution of the resulting schema. At a loaded cost of $125,000 per programmer per year, we estimate a 20-man-year effort will cost $2.5M.

The typical type library is expensive to build; for example, the SAIF spatial type library is about 25,000 lines of C++ code. We expect that over time HAIS will implement a type library of this size at a cost of around $2.5M (based on the industry standard metric of $100 to write and maintain 1 line of code).

Table 3-2 summarizes our recommendations.

Table 3-2: Recommended Actions and Costs
Recommended actions
Cost ($)
Cooperate with SAIF, OGIS, SQL/MM, and POSC in standards efforts.
1.25M
Charter a group led by Earth scientists to define

1. EOSDIS schema.

2. EOSDIS data dictionary.

2.5M
Implement type library.
2.5M

3.3 Hierarchical Storage Management

We expect EOSDIS data to be accessible without operator intervention. This is one way we will be able to reduce staff costs. That means that all EOSDIS data will reside on nearline tape at 2+N sites.

The year 2007 storage hierarchy is shown in Figure 3-1. Hierarchical Storage Management (HSM) cooperates with SQL-* in moving infrequently used data to nearline tape and tape-resident data to the disk cache on demand. Given the long mount times and sequential nature of tape transfers, we expect that the unit of tape-transfer and tape-allocation will be about 100 MB. HSM must manage the movement of these data tiles between the disk cache and the tape robotic devices.

Figure 3-1: EOSDIS Storage Hierarchy in the Year 2007

Integration of a DBMS with an HSM requires that the 2 systems cooperatively manage disk cache for parallel I/O streams. Adding a caching file system (such as multi-resident AFS) between the DBMS and the HSM will complicate the cache control. Direct integration of the HSM with SQL-* is preferred.

There are many HSM products on the market. Members of the alternative architecture team have tried to use several but have found them unreliable. Traditional mainframes have this technology today, but it has not migrated to the open systems world. Future HSM products, such as the High Performance Storage System in development at the National Storage Laboratory, may be sufficiently robust.

In addition, all the products perform file-at-a-time movement of data from tertiary memory to the disk cache. This policy requires any open file to completely reside in the disk cache. Such an architecture is unacceptable for at least three reasons:

For these reasons, EOSDIS should insist on a hierarchical file system that migrates fragments of files, or tiles of data, rather than whole files. We expect that 100 MB will be the traditional fragment size.

It is difficult to find quality HSM systems today. Nevertheless, EOSDIS should not use this as an excuse to write its own. In our opinion, the best strategy is to standardize on a UNIX file system interface (i.e., open-close-read-write). Then EOSDIS should work with 1 or more vendors of hierarchical file systems to produce the needed features. Over the next 3-4 years, a quality product should result. If the chosen vendor(s) does not succeed, EOSDIS can return to the marketplace and select a new vendor(s) of hierarchical file systems.

This strategy will work well because UNIX I/O is already an absolute standard for file systems, which we do not expect to be supplanted in the next several years barring an unforeseen event.

Table 3-3 summarizes our recommendations.

Table 3-3: Recommended Actions and Costs
Recommended action
Cost ($)
Contract with 2 HSM providers to produce needed features in their COTS products.
10M (5M per vendor)

3.4 SQL-* Middleware

The SQL-* middleware box, indicated in Figure 1-4, is the focus of this section. It is responsible for the following tasks:

In this section, we treat each topic in turn, followed by a summary of our recommendations.

3.4.1 Query Optimization, Copies, and Load Balance

SQL-* middleware must locate data in the 2+N site network by interrogating a name service. Current distributed SQL systems (e.g., Oracle Star and Ingres Star) replicate catalog information at several sites, and the query optimizer interrogates one of these sites.

Once the location of each object is known, a cost-based optimizer can evaluate a subset of all possible query-processing strategies to determine the best one. This technology is considered mature by the research community, and commercial optimizers are getting better over time.

However, optimization in an EOSDIS SQL-* context is complicated by several additional factors. First, some of the type extensions may be available only at a few sites. Hence, the optimizer is restricted in its choice of feasible plans. Second, individual sites are not homogeneous in capability, as has been assumed by previous optimizers. Furthermore, some sites may lack the storage necessary to construct intermediate data sets, again restricting the set of feasible plans. Moreover, as noted earlier, it will be desirable to perform portions of any plan in parallel among a subset of the 2+N sites. Opportunities for parallelism vary among plans. Hence, response time as well as the traditional metric of resource consumption must be considered by an SQL-* optimizer.

Next, as noted in Section 1, the raw feed data must be stored at each superDAAC. In addition, superDAACs and peerDAACs may decide to store the results of other nodes in Figure 1-2, resulting in more duplication. The query optimizer, therefore, must decide which of perhaps several copies of an object to use for a given query.

Lastly, in a 2+N distributed system, it will be important for the optimizer to construct query plans that result in an even load balance among the sites. This can be accomplished two ways:

How to teach query optimizers about load balance is not well understood by the research community.

In summary, developing query optimization middleware for SQL-* is a challenging task. Certain aspects are well understood by the research community and implemented in cots middleware, while others are considered research problems.

3.4.2 Lazy vs. Eager Evaluation

For each node in Figure 1-2, middleware must decide whether to use a compute-on-demand lazy strategy or an eager evaluation algorithm. The following analysis gives the framework for making this decision. Suppose the cost of computing the result of a node is Compute_Cost_$, and the cost of storing the result for an hour is Store_$_per_hr. Then if the expected hours to re-request is ReReference_Interval, it is cheaper to recompute than store ifCompute_Cost_$ < Store_$_per_hr * ReReference_Interval

This ignores the cost to the client for waiting for the computation. If we imagine that a clientís time is worth $100/hr, then the equation becomesWait_Cost_$ = Compute_Time_hrs X 100 $/hr

Then, the equation becomes:Compute_Cost_$ + Wait_Cost_$ < Store_$_per_hr * ReReference_Interval

To compute this inequality, one requires an estimate for ReReference_Interval. Each time an object is referenced, the current wall-clock time is recorded in a catalog. The time-weighted average of inter-reference times forms a good estimate for ReReference_Interval.

At each access, the system can evaluate the above inequality. If true, the system should compute the required object, deliver it to the recipient, and discard the object. Otherwise, it should store the object eagerly for the next accessor.

If an object is not referenced for a long time, the system periodically should re-evaluate the inequality for eager objects to see if they should be discarded.

This analytical framework can be extended easily to accommodate

ï Finite limits on storage.

ï Compute costs, which depend on whether input data in Figure 1-2 are cached or must be computed.

In the interest of brevity, we omit details.

3.4.3 Lazy and Eager Evaluation

In this section, we indicate how to optimize the early versus late processing decisions. To illustrate our approach, we use a processing step that regrids AVHRR data to a different grid system than the one used for the original data. The original data is in the table:create RASTER(time=TIME, location=Box, band=Spectrum, data=Array2D)

and the processing step performs the following SQL-* computing:select into NEW-RASTER(time, location, band, regrid(data))from RASTER

Consider a subsequent query for regridded data, such as
select clip(data, RECTANGLE)from NEW-RASTERwhere band=BANDand time=TIME

To support lazy evaluation of the regridding step, the following SQL-* view must be defined:
create view NEW-RASTER as select time, band, location, regrid(data) from RASTER

Views correspond to virtual tables. No data but only the semantic definition of the view is actually stored for NEW-RASTER. Later the user issues the above query. In this case, NEW-RASTER does not actually exist, and the DBMS execution engine must perform query modification on the user query by substituting it into the view definition to construct the following query, which is actually run:
select clip(regrid (data), RECTANGLE)from RASTERwhere band=BANDand time=TIME

In summary, the view system present in all SQL systems is the mechanism to support lazy evaluation.

To support eager evaluation, the Project Sequoia 2000 team has three alternative suggestions:

We discuss each of these in turn.

To perform eager evaluation, we need only specify the following trigger:on insert into RASTER theninsert into NEW-RASTER select new.time, new.location, new.band, regrid(new.data)

This trigger watches for the insertion of a new row into the RASTER table, then triggers a corresponding insert into the NEW-RASTER table. Specifically, it appends the time, band, and location from the inserted record as well as the regridded data. In this way, the derived data, NEW-RASTER, is kept current with RASTER by using a trigger.

A user can now issue a query for regridded data, such asselect clip(data, RECTANGLE)from NEW-RASTERwhere band=BANDand time=TIME

This query will be satisfied quickly by the DBMS from data stored in NEW-RASTER. As such, eager evaluation makes use of a trigger to generate derived data at the time the original data becomes available. This solution is desirable in that one can switch between eager and lazy evaluation without impacting the queries that users will write. In this way, users are insulated from this optimization decision, and it can be changed as conditions warrant. However, this solution does have the following disadvantages:

In contrast, if each trigger is run as a separate transaction, disconnected from its predecessors, there is no easy way to restart a trigger that fails because there is no enclosing application program to specify reactivation. This will lead to dangling computations, which are incomplete.

What would be desirable is a good work flow system to which Figure 1-2 could be defined. This system, which we consider the second approach to eager evaluation, would optimize the eager versus lazy decision for each node, specify appropriate views for lazy nodes, and supervise execution of the eager pipeline, including semantically correct failure recovery and recomputation if the definition of an eager node changed.

We are unaware of a COTS work flow system with these properties. Therefore, HAIS could build one, perhaps in conjunction with a DBMS partner. We are not enthusiastic about this solution for the following reasons:

A third approach falls somewhere between the first two: Use triggers in a local DBMS to fire when an object becomes available. The action part of the trigger is not an SQL-* statement as in the first approach but rather a user-defined function that sends an appropriate message to a transaction (TP) monitor (e.g., Tuxedo or Encina). This message contains the SQL-* code to execute, and the TP monitor can be responsible for its reliable execution, even in the presence of failures.

Whichever suggestion is followed, we expect HAIS will have to write considerable custom code to make any of these systems workable and easy to use. This is factored into our recommendations at the end of this section.

3.4.4 Interactive vs. Batch Processing

As noted earlier, some queries to EOSDIS will run for days, even weeks. Such queries should be batched with other queries from hell and processed together in a batch using a single sequential pass of the stores at appropriate sites. It is the job of the SQL-* middleware to detect whether a query should be run interactively or through this batch mechanism. This query optimizer produces an estimate for the CPU and I/O resources needed for each query. If the I/O requirements are prohibitively large, for example requiring 100,000 random tape reads, the query should be queued for the batch system and a suitable message sent to the requestor. If done in batch, the middleware must support collecting together, executing, and disassembling the appropriate answers for such batches.

3.4.5 Summary

Several object-relational SQL systems exist (UniSQL, Illustra, ODI, and Objectivity). Several parallel-distributed DBMSes exist (AT&T, IBM, INGRES, Oracle, Sybase, and Tandem). All provide distributed query optimization, data replication, distributed transactions, and parallel execution. However, none yet combine all these features with support for their object-relation products or load balancing.

Certainly, AT&T, Oracle, Informix, IBM, Sybase, and Tandem are investing heavily in parallel database systems. But these efforts are not connected with the object­oriented efforts in the same companies. Moreover, no distributed DBMS contains suitable work flow management, distributed triggers, or early-vs.-late optimization.

Therefore, we recommend that NASA contract with 2-3 COTS SQL vendors to accelerate the timetable of their commercial offerings to produce SQL-* middleware. In all areas except work flow management, NASA should be able to use this technique to get a usable product in the appropriate time frame. Contracting with multiple vendors will lower the risk of failure as well as allow perhaps multiple implementation approaches. We expect this approach to be far less risky and far less costly than if the EOSDIS contractor were to write a complete proprietary SQL-* middleware system from scratch, an endeavor that we estimate at 500,000 lines of code costing $50M.

In addition, we expect HAIS will have to write proprietary work flow code using one of the three approaches we outlined. We expect this code will be 25,000 lines and cost $2.5M.

Table 3-4 summarizes our recommendations.

Table 3-4 Recommended Actions and Costs
Recommended actions
Cost ($)
Contract with 2-3 vendors to produce SQL-* middleware.
10-15M
Design and construct an eager pipeline system.
2.5M

3.5 Distributed and Client/server Systems Issues

EOSDIS must define a protocol though which client programs can communicate with SQL-* middleware and middleware can communicate with local and remote SQL-* servers. In our architecture, SQL-* queries go from client to SQL-* middleware to local server, and results are returned from local server to middleware to client. Obviously, the EOSDIS protocol should be designed with SQL in mind.

3.5.1 SQL-* APIs and FAPs

There are two general approaches to SQL client-server protocols:

ANSI SQL and Microsoftís Object Database Connection (ODBC) are examples of APIs that define how an application program issues SQL queries from a C (or other third-generation language) program. A vendor who supports ANSI SQL or ODBC is free to construct his own (proprietary) on-the-wire protocol. He must additionally write a library that maps run-time function calls to the correct on-the-wire utterances and sends them to a server. The server accepts on-the-wire SQL statements in this vendor-specific on-the-wire protocol and executes them.

If all vendors support the same API, the programmer has portabilityóhe can run his program using any vendorís call library.

A standard API, however, does not necessarily give interoperabilityóthe ability to connect 2 or more different database systems into a single distributed database in which an application can make calls to any part of any database anywhere. Notice that in the API example above, the private vendor protocol implied that there was vendor-specific code at each end of the wire.

The ability to combine 2 or more different databases requires that they communicate in some agreed upon language. The ISO Remote Database Access (RDA) standard defines a FAP used for interoperability. X/Open and the SQL/Access Group have built their interoperability plan on the use of RDA (and OSI/TP for transaction atomicity). RDA defines how SQL messages must be formatted between clients and servers and how SQL data can be returned from servers. By demanding that vendors support the RDA protocol, a user is free to access any database, even ones foreign to his native vendor. RDA lacks the object-oriented extensions to SQL-3 or SQL-*. Adding network-independent type libraries to RDA may take a long time. As a result, there is no standard on-the-wire protocol in sight.

In addition, the API or FAP standards have not achieved their goals of portability and interoperability because all vendors implement supersets of the standards. These vendor-specific extensions tend to be very useful to applications that use them to create non-portable non-interoperable applications. Hence, database products are advancing much more rapidly than the standards process.

Instead, interoperability is typically provided via vendor-specific gateways. A gateway maps one vendorís version of SQL into another vendorís version. Typically, the gateway resides on the client side of a client-server connection and performs its translation on the client. In addition, the gateway typically has an agent at the server site that generates calls in the API of the target DBMS and then includes the library implementing this API.

The problem with gateways is that N products gives rise to N*(N-1) gateways. This number is unmanageably large, which is why there is a strong trend to reduce N by picking a single vendor or insist that all gateways talk the same language, reducing the number to N gateways.

In an ideal world, everyone would speak Esperanto, and there would be no gateways. Realistically, EOSDIS should expect to have gateways to legacy database systems and gateways among the few database engines it selects for the 2+N DAACs. The construction of these gateways is the responsibility of the DBMS vendors. If they lack appropriate gateways, they should not be considered candidate storage engines.

SQL-* is guaranteed to need gateways to achieve vendor independence. Moreover, if key features are not in the eventual SQL-3 standard, gateways will still be required to support these features after the turn of the century.

The net result is that interoperability will be provided only through gateways and not through an FAP. Since gateways interact with the API, it is clear that SQL-* should be defined as an API. Gateways will then map to vendor-specific on-the-wire representations. EOSDIS should track the progress of SQL-3 and RDA but not depend on them. It may well be that de facto standards promulgated by Oracle, Microsoft, IBM, or Sybase will become the real standards, thereby supplanting the function of RDA.

As a result, EOSDIS should pick, as its SQL-*, the SQL product that comes closest to meeting EOSDIS object-relational needs. Moreover, EOSDIS should adopt the API used by that vendor as its standard definition of SQL-*. The vendor must run on all popular platforms, thereby ensuring that servers can be deployed on many machines. In addition, the vendor must provide gateways to the other popular DBMS engines (Oracle, Sybase, Informix, and DB2) so that others can freely implement their type libraries on the most appropriate engine.

3.5.2 Other Client-server Protocols

Here we briefly discuss several other approaches to client-server protocols, including Z39.50, CORBA, and NCSA Mosaic/HTTP.

Z39.50 is a protocol used in library automation. We rejected the idea of building EOSDIS from a Z39.50 base because Z39.50 is a cottage industry with an arcane design. It lacks a general data model and data definition language. It lacks most of the relational operators and type extensions, parallelism, or distribution.

Z39.50 is oriented toward searching bibliographic records. It supports the concepts of keywords and proximity searches, which are built into the protocol.

To adapt Z39.50 for arrays, spatial data, and other type libraries would be a major design task. EOSDIS would have to define a type system and query language and then extend Z39.50 with it. In our opinion, a more direct approach would be to add the capabilities in Z39.50 as a text type library to SQL. Oracle has done something like this as an extension of their SQL product.

Building from Z39.50 would be proprietary to the EOSDIS contractoróthe worst of all worlds. The contractor would have to construct gateways to any DBMSes of interest. Moreover, client programs would have to be adapted to this protocol, presumably at the contractorís expense.

An SQL-*-to-Z39.50 gateway could be written to support clients or servers that provide only a Z39.50 interface if it becomes necessary to support such a server.

On the other hand, if ISO/ANSI/FIPS standard API and FAP are chosen, no gateways will be required to support new EOSDIS tools and engines coded to the standard. Moreover, vendors would be motivated to make other tools run on their engines and support their tools on other engines. Standard APIs and FAPs put the gateway burden on the vendors at no expense to EOSDIS, resulting in substantial leverage. This leverage is lost with a contractor-proprietary protocol.

CORBA (Common Object Request Broker Architecture) is a client-server request-reply protocol defined by the Object Management Group (OMG). It is the successor to DCE-RPC. It is a standard way for a client to invoke a server. Unlike DCE, CORBA allows the client to decide the procedure name and parameters at run time. DCE-RPC requires the client to know the procedure at compile time and so is not useful for resource discovery applications. Using CORBA in EOSDIS would require defining new kinds of CORBA objects, namely an SQL-* QUERY and SQL-* RESULT because CORBA does not have a database interface. Such an extension would be non-standard and contractor-proprietary.

EOSDIS servers will often be returning very large objects to clients. Therefore, they would like a stream protocol, whereby the server streams data to the client, with flow control to keep the client from being overrun by a faster server. Unfortunately, CORBA is a stateless RPC-like protocol and includes no stream processing. This deficiency could be overcome by defining an EOSDIS-specific stream protocol on top of CORBA but, again, this extension would be non-standard and contractor-proprietary.

A third serious problem with CORBA concerns CORBA gateways. Because CORBA is an API protocol, each vendor supporting CORBA is free to use his own on-the-wire protocol. However, CORBA vendors have not written the gateways that are required to map their CORBA implementation to the on-the-wire representation expected by a different vendorís CORBA. As a result, no two implementations of CORBA interoperate (the recent IBM-HP interoperation demonstration used non-product non-standard software). A Sun Microsystems client using CORBA cannot interact with an HP server using CORBA. Hence, current CORBA is useless as an interoperability vehicle.

A last problem with CORBA concerns work flow. As noted in Section 3.4, a key feature of our architecture is reliable re-execution of eager commands in the event of a failure. The current version of CORBA has no such transactional feature.

In short, the functionality of CORBA is not a good technical fit to EOSDIS needs, and correcting the problems would yield a proprietary protocol, which would sacrifice leverage and not foster interoperability. In time, these problems may be fixed. CORBA should be watched by EOSDIS and reconsidered when the following exist:

We expect these issues will take several years to be resolved.

The phenomenal acceptance of NCSA Mosaic dictates that EOSDIS must support Mosaic access to its data. There are two ways that Mosaic can be supported. First, Mosaic documents can have embedded references (Uniform Reference Locators, or URLs) to documents stored at other sites. To allow EOSDIS to be a Mosaic server is straightforward: A front-end program must support the HTTP protocol and map URLs to local database objects, and invoke procedures to access those objects with the parameters passed by the invocation. This mapping can be done by a gateway using SQL-*. Mosaic has a primitive query mechanism that can request subsets of the data. An HTTP-to-SQL-* gateway can interpret client requests and return results for display.

Whatever vendor(s) EOSDIS selects will provide a Mosaic/HTTP interface with these capabilities. It should be noted, however, that there is no data security available in the current Mosaic client-server protocol, although improvements to Mosaic should correct this deficiency over time.

Besides HTTP (and perhaps Z39.50), there may be other desired gateways (e.g., NFS, Archie, and MIME). In addition, there will be gateways to various legacy DBMSes. In EOSDIS there will be some repositories that support a query language less functional than SQL-*. These include all the Version 0 processing centers. EOSDIS should decide what legacy gateways are required, then find a vendor willing to construct them. The EOSDIS contractor should not get into the business of writing gateways as this functionality is best performed by companies that specialize in gateways, such as DBMS, 4GL, and middleware vendors.

In our opinion, the easiest way to build such gateways is to use the source code for an SQL-* engine. The current vendors have written gateways to the popular relational products, and it is reasonable to use the same technology to build gateways to other kinds of systems. Hence, one of the SQL-* vendors may be the best candidate for constructing legacy gateways.

Table 3-5 summarizes our recommendations.

Table 3-5: Recommended Actions and Costs
Recommended actions
Cost ($)
Choose SQL-* protocol.
0
Provide gateways to popular DBMSes.
Hopefully 0
Provide gateways to legacy DBMSes.
Unknown (depends on what is needed)

3.6 Operating Systems, Networking Software, and Programming Languages

There is surprising consensus that EOSDIS should be based on open systems available from multiple vendors. Loosely translated, this means UNIX, the X Windows System, DCE, Fortran, C, C++, and TCP/IP. It excludes MVS, VMS, MacOS, MS-Windows, NT, NetWare, SNA, DECnet, and PostScript.

The exclusion of MacOS, all the Microsoft products (MS/DOS, Windows, and NT), and PostScript is clearly unacceptable. The majority of desktops will likely be running Microsoft operating systems. Also, we expect that the Microsoft OLE type libraries will dominate the industry. Clearly, PostScript will continue to dominate the printing industry.

So, the formula is not so simple. Fundamentally, EOSDIS does not want to be locked into a single vendor or product. It wants to be able to migrate to new vendors and new technologies as they emerge.

3.6.1 Operating Systems

Today and for the next 3 years, UNIX is a safe bet for an EOSDIS operating system. There are UNIX variants available for all the major hardware platforms. There is a rich set of database, networking, system management, and program-development products on these UNIX systems. The obvious alternative is NT, but it is immature and has relatively few products today. It is likely that NT will ultimately support UNIX and DCE personalities and become UNIX-branded by the X/Open consortium that now owns the name. Indeed, today NT has the best DCE-RPC implementation. Given that, it may well become the UNIX of choice.

UNIX file systems are fairly standard. They are all moving towards 64-bit file addresses (Exabyte file sizes). To get the best performance, most database systems use the raw-disk interface (rather than the cooked file interface) of most UNIX systems. These interfaces have become fairly standard as well.

Many UNIX vendors have been late to support multiprocessors (e.g., this capability will soon be added to AIX), but all now promise kernel threads and full SMP support by mid 1995.

Support for clusters of multiple shared-nothing processors (analogous to Tandem, VMScluster, or Sysplex) is still a promise from each vendor. Products are beginning to emerge, but standardization will probably not start for a year or two. This means that EOSDIS will have to work with a particular vendorsí software to manage a processor array if present at a super- or peerDAAC.

3.6.2 Networking Software

The explosive growth of the Internet, and the convergence of computers, communications, and entertainment are creating enormous demand for network software and bandwidth. Today the Internet is transferring terabytes per month. Demands for videoteleconferencing and NCSA Mosaic will likely fuel rapid improvements.

EOSDIS therefore can piggyback on the Internet. The nation will have a high-speed backbone, and there will be software and hardware to deliver data at megabyte-per-second rates to Earth scientists&IACUTE; desktops. Indeed, by the end of the decade, they will likely have slow 155-Mb/s data rates to their desktops. Many are predicting that similar rates will be available to offices, classrooms, and homes.

This bandwidth will not be free, but it will be inexpensive enough to allow individuals to chat with one another via video streams or to watch a sports or news event.

We conclude that EOSDIS can surf the networking wave and benefit from network innovations being created for the Internet by commerce and the entertainment industry.

As noted earlier, EOSDIS should use TCP/IP and DCE as its lowest common denominator network software. Also, CORBA, in its present form, should not be considered. Moreover, EOSDIS should watch the Microsoft networking strategy carefully.

3.6.3 Programming Languages and Environments

There have been many programming language fads. High-level languages, structured programming, relational databases, Ada, and now object-oriented programming have all been touted as silver bullets that would yield huge savings and productivity increases. Certainly, each of these developments has been an advance, but none has changed the fundamental nature of programming. Spreadsheets, GUI-query interfaces, and other visual programming tools have had the greatest impactóthey have eliminated programming or at least made it something the end user could do.

Our concept of EOSDIS relies heavily on reducing the staff needed to deliver data to the scientist or data subscriber. Our premise is that the GUI query tools and programming environments appearing on desktops will be able to send SQL-* requests to the DAACs and receive answers without any human intervention.

EOSDIS is not chartered with inventing this client software, but it will have to create viewers as part of each type it invents (display is one of the standard methods of a type library). Beyond these viewers, we expect the rest of the client software will come from browsers invented by others. NCSA Mosaic is one such browser; Microsoft Access is another.

But how shall EOSDIS implement the class libraries it invents? The Earth Science community has grown up with Fortran and associated tools. The Computer Science community has grown up with the C language and associated tools. EOSDIS will be a combination of these 2 languages. It will be possible to write type libraries in either language.

SQL-* is an object-relational database system, which has object-oriented type libraries at its core. Consequently, we believe that C++ will be the real EOSDIS language. Type libraries will be installed in SQL-* with C++ wrappers that translate from C++ to Fortran or C.

There are many competing object standards. The object management group (OMG), the Object Data Management Group, SQL-3, C++, Borland, Microsoft, Apple, Sun Microsystems, and HP each have one, and IBM has several. All these standards have a slightly different notion of an object. Since all other objects inherit their behavior from the root object, it is difficult or impossible to move an object from one vendor (standard) to another. There is little that EOSDIS can do about this beyond waiting and watching. By 1996 it will have to pick a standard root object.

Table 3-6 summarizes our recommendations.

Table 3-6: Recommended Actions and Costs
Recommended action
Cost ($)
Monitor object standards as they unfold.
375K (1 person for 3 years)

3.7 System Management

The superDAACs will have hundreds of processors and network lines, thousands of disk and tape drives. We expect to run each superDAAC as a lights-out operation with small operations staff (2 persons 24 hours/day) and a small administrative staff that controls the system growth and change.

As mentioned before, the work flow system is responsible for automating the push processing for standard data products, automating the service of interactive requests, and automatically scheduling eager evaluation. User questions about the data will be serviced by the EOSDIS schema. Failing that, questions about data quality and lineage should be sent to the team that generated that data product.

Thus the DAAC day-to-day management is just change management. The major changes to each DAAC would be

The first two tasks are operational, the second two are administrative. To have a small operations staff, EOSDIS requires that the hardware and software be reliable. It also requires that when a module fails, it be self-diagnosing and alert the operator about the problem.

The proliferation of networks has created tools and standards that allow systems to scale to thousands of nodes and still be manageable. A standard product suite is emerging around the Simple Network Management Protocol (SNMP) and the NetView product suite from HP, IBM, and DEC.

The idea is that each hardware and software module has a Management Information Base (MIB) that describes its status and an agent that services requests to read and write the MIB. A System Manger, like NetView, can poll the status of each MIB to collect a picture of the current system state. Similarly, a system manager can request that the MIB agent change the module state (e.g., start, stop, reset, etc.). The new version of SNMP adds security and events, allowing the agent to authenticate and tell the manager when something changes (this avoids polling).

MIBs have been defined for most hardware components, operating systems, network links, routers, bridges, and database systems. These MIBs are widely implemented, and they are becoming a standard part of software and hardware subsystems.

But MIBs are only 90% of the story. One also needs an operations center to monitor the MIBs, detect anomalies, diagnose them, and drive the repair process. There are many such products, but NetView is a multi-vendor solution. It polls the MIBs and displays current status. It fields events from the MIBs and alerts the operator. Each MIB class has a viewer that walks the operator through the diagnostic and repair process. For software modules, this is typically the administrative interface to that subsystem.

EOSDIS should evaluate these products, pick one, and insist that software and hardware components in a DAAC have a good SNMP interface.

Table 3-7 summarizes our recommendations.

Table 3-7: Recommended Actions and Costs
Recommended action
Cost ($)
Buy a COTS system management tool suite.
200K (100K per superDAAC)

3.8 Summary

In summary, we recommend that EOSDIS do the following:

  1. Adopt a database-centric approach to the DAACs.
  2. Pick an interim SQL-* by selecting the API used by the vendor with the closest approximation to SQL-*.
  3. Cooperate with SAIF, OGIS, SQL/MM, and POSC in standards efforts.
  4. Design and implement Earth Science-specific type libraries using SQL-*.
  5. Design and implement an SQL-* schema for EOSDIS data.
  6. Design and implement a data dictionary for EOSDIS clients.
  7. Encourage the integration of SQL-* systems with hierarchical storage managers.
  8. Encourage the development of parallel and distributed SQL-* systems.
  9. Design and build a work flow system to manage the eager evaluation pipeline, including the eager versus lazy optimization decision.
  10. Choose an SQL-* protocol.
  11. Contract with 1 or more vendors to construct gateways from SQL-* to required legacy systems (e.g., version 0) and other required local DBMSes.
  12. Monitor object standards.
  13. Adopt an SNMP-NetView management strategy, requiring all hardware and software modules to have an SNMP interface.