Repositories: ATG's Object Relational Mapping

Introduction:

ATG repository is Object Relational Database Mapping service for Java or J2EE Environments. Repositories are used with ATG Dynamo core services and can’t be used with Other Frameworks.

Object Relational Mapping: 

Object relational mapping is a technique for converting data between incompatible type system in object-oriented programming languages. Its like a "virtual object database" that can be used from within the programming language.

Object Persistence Implementation:

Most objects cease to exist when they go out of scope. This may be when the function in which they were created terminates. It may be when the container in which they reside is deleted. At any rate, they can be expected to disappear when the program exits. Persistent objects are those which survive between successive invocations of the program. A classic example of such an object is a database record.

Few Issues to Handle while doing Object Persistence implementation:

      1. Object Relationship Mapping Architecture

      2. Code Generation

      3. CRUD Implementation (Create, Retrieve, Update, Delete)

      4. Query

      5. Transaction

      6. Cache

1. Object Relationship Mapping Architecture:

    Repository is composed of items, items is composed of item descriptor, item descriptor usually corresponds to a table in data store. Item Descriptor is a Java Bean, but not regular static Java bean. It is Dynamic Bean. 

    Dynamic Bean contains properties, and two methods, getProperty and setProperty. Properties are available in run-time only, and must be defined in a definition file. Definition file is a XML file, table fields, tables and table relations are associated with properties in the definition file. Because item is a java bean, it can be used in JVM and don't have to prepare it a static java class

* IdGenerator is one of the ATG core service and is responsible for defining how IDs need to be obtained.

Table relations are the most complex part in any mapping file.Repository use Collection, including Set and List, to defined one-to-many relations.  Repository use component-item-type to tell Ids in collection are associated with what item descriptor, the ID is used as key in program to retrieve and cast out the item descriptor associated. 

2. Code Generations:

    Code generation in DAO (Data Access Object) pattern is to create java bean based on or mapping definition file

    

    ATG Repository do not have code generation tools, and not necessary to have it because of the nature of item. ATG prepared utilities to export DDL for users to create schema, syntax is startSQLRepository -outputSQLFile xxx.ddl or use print-ddl tag in Admin console to export DDL. Usually this is enough for initial setup in ATG platform.

3. CRUD Implementation:

   Persistence Storage has four basic functions, CREATE, RETRIEVE, UPDATE, DELETE. Persistence tools handle CRUD in the same way on single table, but some difference on associated tables – hierarchical properties.

  Repository use the attribute “cascade” to define CRUD on parent and child tables.Repository cascade attribute values contains “insert”, “update”, “delete” and can be used as combination. It is required to add item in child table at first before add item id in link table, so does in delete, cascade is not to handle forward reference in Repository. ATG Developers must be aware of the subtlety.

4. Query:

    

   Repositories uses Repository Query Language to provides functions of querying objects and querying database.

   Repository RQL prepared more customized queries such as ComparisonQuery, PatternMatchQuery, TextSearchQuery, IncludesQuery, IncludesAnyQuery. RepositoryView is to build and execute query, corresponding to Criteria.setReadOnly(true), Query.setReadOnly(true), Session.setDefaultReadOnly(true)(immutable Entity) in Hibernate

5. Transaction:

  A transaction refers to to a process where each of the steps must complete and if any step fails than the process roll back. A transaction can complete in two way COMMIT or ROLLBACK.
For example

begin transaction 
          debit checking account
          credit checking account
          update history log
commit transaction

  ATG Transaction, including atg.dtm.TransactionDroplet, atg.dtm.EndTransactionDroplet, atg.dtm.TransactionDemarcation, is core ATG service and ready for instantiate from local application, such as dsp page (dsp:beginTransaction, dsp:droplet transAttribute), form handler(atg.droplet.TransactionalFormHandler), or programmatic demarcation in customization class. 

 ATG JMS service(Patch Bay) had its own transaction management(container-scoped, session-scoped). ATG had already defined transaction boundary(Demarcation) based on the scope of application/objects persistence. ATG Transaction mechanism and API are highly customized for your need and convenient to use. ATG support six type of Transaction Demarcation Modes such as  

REQUIRED, 

REQUIRSNEW, 

SUPPORTS, 

NOTSUPPORTED,

MANDATORY,

NEVER

6. Cache:

  ATG SQL Repository Caching has two types, item cache and query cache. 

Item cache stores item bean,

query cache stores repository id only.

Repository support cache modes such as “simple”, “inherit”, “locked”.

  Locked caching used in distributed environment where multiple instance and servers can access cached items, and cached items are managed by ServerLockManager, and client side item cache access is managed by ClientLockManager.

  ATG provide strategic level of repository item cache management mechanism, including the three mode, “distributedTCP”, “distributedJMS”, “distributedHybrid”. 

  Distributed TCP caching is implemented by SQLRepositoryEventServer where listener catch client cache access event and delegate call to ServerLockManager in primary cache repository. 

  Distributed JMS caching is implemented by GSAInvalidatorService and GSAInvalidatorReceiver in which RemoteObject and RMIService handle client call to invalidate specified repository item cache. Distributed Hybrid stands for “locked” caching mechanism + synchronization among subscriber client server, GSAClientManager and GSAServerManager handling synchronization request(GSACacheEvent), the requests are delegate to ServerLockManager in local/primary cache instance.

  ATG Repository support version repository, secured repository, composite repository(multiple data store connected). Content Repository support contents from both data store and file system. Repository Loader could load xml file into versioned/non-versioned database. ATG prepared Web Service API to access and operate on repository, and we are not necessary to hit database directly to CRUD records.

  In summary, ATG repository is a complete and powerful object persistence service and even more, some outstanding features such as derivative properties, multi-value properties, convenience of transaction, file system support, service API, etc., had significantly improve the productivity of development and accommodate the most advanced technical demand in today's web environment. ATG repository has the potentials to be an independent commercial software if ATG make it to be fully portable one.

Repository Enables Data anywhere architecture:   

             ATG’s Data Anywhere Architecture (DAA) meets all of these challenges. DAA gives developers a single API, called the Repository API, for using data resources in their applications. Behind the Repository API, DAA insulates and abstracts application developers from the specifics of the data source, so much so that the nature of the underlying data source may completely change without major impact. For example, customer data may reside in a SQL/JDBC database today, but will move to an LDAP directory in the future. DAA could handle this without having to touch any code within the application. The fundamental construct in the DAA is a ‘Repository’. A Repository is a logical view of a data resource (or resources), and to a developer, manifests itself as a set of JavaBeans to be used within their application. Like everything else in an ATG application, Repositories are represented as Nucleus components.

The Repository is described in a Repository Definition XML file, which holds all appropriate information about the data’s physical location and how it is mapped to the logical view. The DAA consists of three primary Repository types for data access and manipulation.

SQL repository – A SQL Repository presents a logical view of data stored in a relational database, accessed through JDBC. The Repository definition file defines how the databases, tables, rows, and columns of a relational database are presented through the Repository API. It also defines the item caching strategy to be employed to optimize database read/write performance.

LDAP repository – An LDAP Repository presents a logical view of any data source that has an LDAP interface, typically used to store user data. The Repository definition file defines how the hierarchical structure and contents of an LDAP directory are presented through the Repository API.

Integration repository – In some cases, data sources may not be directly accessible, or information may be returned by an application rather than directly from a database or directory service. The Integration Repository presents a Repository API in front of some remote application processing. For example, an Integration Repository may be implemented to facilitate integration with SAP’s BAPI interface, or to retrieve information through the execution of a Web Service SOAP call. The Integration Repository is an open architecture into which these specific integration technologies can be plugged, while still presenting the same Repository API to the application developer. It also gives developer sophisticated data access and caching strategies to protect the application from remote latency and downtime.

In addition to the primary types of Repository mentioned so far, there are two types of ‘overlay’ repository types that can be used.

Secure repository – A Secure Repository introduces application level security and access control to the data being accessed and manipulated. Working with ATG’s Security Management Framework, varying levels of security can be defined on the Repository contents, all the way down to individual data properties. Access Control Lists (ACLs) are written to describe the different levels of access that are provided to ATG’s User Model, which itself provides a rich structure to model user, organizational hierarchies and roles.

Versioned repository – A Versioned Repository introduces a versioning mechanism to one of the other primary Repository types. It provides all of the required tools to maintain, version and roll-back versions of a Repositories contents. Any existing SQL Repository may be turned into a Versioned Repository through additional configuration files. The Versioned Repository architecture is heavily used by ATG’s Content Administration product, but the versioning features are open for any other type of application usage that may be customer specific. Versioned Repositories integrate closely with ATG’s workflow capabilities that reside in the ATG Adaptive Scenario Engine. A Composite Repository is the final construct that can be especially useful for building applications requiring access to data in multiple data sources and formats.

Composite repository – A Composite Repository represents an aggregate view over other repository types, or over other composite Repositories (although one should not create too many layers of Composite Repository). The most common use of a Composite Repository is where a businesses customer data is distributed over multiple SQL databases and an LDAP directory, but a Web application wants a ‘single view of the customer’ to reduce application complexity.

A Composite Repository provides some welcome simplicity.

To ensure scalability of Web site usage of SQL database, the DAA provides sophisticated caching and cache invalidation mechanisms for SQL Repositories.

DAA provides all of the necessary tools to manage and flush caches at the repository item level. There are also mechanisms for managing distributed caches

and cache invalidation via JMS or TCP/IP. All in all, ATG’s Data Anywhere Architecture provides a rich, robust, and highly scalable set of tools to facilitate the use of enterprise data resources, while providing a loose coupling approach between data source and application.

Repository Technically:

-Profile, Content, Commerce repositories is example for these repositories.

*Repository Item : (atg.repository.RepositoryItem) : must have id +/- any other properties.

Some droplets use these repositories:

-RQQueryForEach & RQQueryRange

-ItemLookupDroplet & RepositoryLookup..

-How to define repository XML:

<gsa-template>

<item-descriptor name=”….” ….>

….. description here…..

</item-descriptor>

</gsa-template>

-Elements attributes could be :

1) Transient: not persisted in DB.

2) Persisted: stored in DB , you need to provide the DB table/column.

the table relation type could be “primary” , “auxiliary” or “multi”.

3) Derived: Calculated with each request, you need to provide derivation

class that extends DerivationMethodImpl class.

Properties could be:

-Simple : string, integer..

-Enumerated: option value=x code=x where the code is what stored in DB.

-Items: refer to other items.

Also it could be:

-Single Values

-Multi-values: Array, Set and Map of other objects.

**Inheritance:

It is supported in XML…

Base —> item-descriptor ….sub-type=class

Child –> item-descriptor ….super-type=class

- You can also use layering to override , replace , remove elements according to your needs…

just specify the xml-combine attribute with either:

-”remove”, “append” or “replace”.

Some repositories already defined and you can extend the repository/modify by having a layer (or layers) that define that repository definition file as:

userprofile.xml (for User Repository)

customCatalog.xml (for Commerce catalog repository)

priceLists.xml (for commerce price lists)

NB: You can cascade in relations by cascade=”delete”

NB: You can group some properties for better performance: property=first group=full_name and property=last group=full_name

NB: You can use id generator to generate Ids for repository by :

<item-descriptor name=xxx id-space-name=”unique name for id generation”

This is either auto-generated or explicitly configured in /atg/dynamo/sevices/idspaces.xml

**Object Types:

1) Repository: used to get repositoryItem (with its id) and to obtain repositoryView

–> Methods : getItem() and getView()

2) RepositoryItem : represents the individual item of the repository.

–> Methods: getRepositoryValue()

3) RepositoryView : to execute queries to return array of repositoryItem.

–> Methods: executeQuery()

4) MutableRepository : methods to insert/update/detlete repostiryItems

–> Methods : addItem(), createItem(), updateItem(), removeItem() and getItemForUpdate()

5) MutableRepositiryItem : used to change/set values of repositoryItem.

–>setPropertyValue()

Example: To Update

MutableRepositoryItem mutableItem=mutableRepository.getItemForUpdate(id,”Article”);

mutableItem.setProperty(…., …);

mutableRepository.update(mutableItem);

Example: To create

MutableRepositoryItem mutableItem=mutableRepository.createItem(“Article”);

mutableItem.setProperty(…., …);

mutableRepository.addItem(mutableItem);

**Using RQL queries: either as statement (String) or named query (in repository XML)

RepositoryView myView=repos.getView(“Article”);

Object rqlParams=new Object[];

rqlParams[0]= “Osama”;

RqlStatement statement=RqlStatement.parseRqlStatement(“name = ?0″);

RepositoryItem[] artcileList=statement.executeQuery(myView, reqlParams);

….. loop over the results……

In XML:

<item-descriptor name=”Article”>

….

<named-query>

<rql-query>

<query-name>MyQuery</query-name>

<rql> name = ?0 </rql>

</named-query>

….

</item-descriptor name=”Article”>

And in the code:

NamedQueryView view=(NamedQueryView) myView;

Query namedQuery=view.getNamedQuery(“MyQuery”);

….

ParameterSupportView pView=(ParameterSupportView)myView;

RepositoryItem[] artcileList=pView.executeQuery(namedQuery, rqlParams);

….

Repository Simplified:

In ATG repositories are ATG components with a configuration file and a class. ATG provides some existing repositories which can be customized and we also can create repositories from scratch.

There is a XML file which defines the tables and we also can club multiple tables using XML mapping in to a single java object.

A fine Example from http://learnoracleatg.blogspot.in/

consider the below table structure:-

TABLE STUDENT: (Has 4 columns)
1. student_id
2. name
3. age
4. address_id

Table ADDRESS: (Has 5 columns)
1. address _id
2. house_number
3. street_name
4. state
5. pin_code

Now, "STUDENT" table has student_id, name, age and address_id. This address_id is linked to another table, "ADDRESS" (which also has address_id). Now for this address_id, we have a house_number, street_name etc.
These are basically two tables with a single relationship (using address_id column).

If you have worked on JDBC (not required for this), you can understand that you'd have to fetch details for a particular student_id from "STUDENT" table, manually map the address_id to "ADDRESS" table via JAVA CODE and again fetch the details from this table (street name, state etc. etc.)

A repository saves you the pain for all this "linking" via java code, and provides you a single object (which you can configure), from which you can fetch all details using simple repository API .

Therefore, what the repository does for us is, it creates a wraps all the columns we require from a set of linked tables into a single JAVA OBJECT DEFINITION for all these relations, so that it is easy for us to access the database.
This JAVA OBJECT DEFINITION is called "item-descriptor".
Also, it is we, who define what properties this "item-descriptor" must contain and which column of a table its properties map to.

Therefore, this item-descriptor is basically an XML block (written in an XML file), which contains the mapping of your database columns to the properties of the JAVA OBJECT you want to create.

                                   

This concept is known as Object-Relational-Mapping (ORM), which maps relations in relational database to Objects.

The Repository Component: 

* Repository is a component which is supported by XML. Below are the steps we can create repository..

   Two most important things are Repository Component and referred Repository Class.
*Class used by repository is atg.adapter.gsa.GSARepository

* The configuration file

$class=atg.adapter.gsa.GSARepository
$scope=global
repositoryName=CustomRepository

*Now the configuration file with supporting XML.

$class=atg.adapter.gsa.GSARepository
$scope=global

repositoryName=MyNewRepository
definitionFiles=<YOUR_CONFIG_PATH>/<YOUR_XML_FILE_NAME>.xml
XMLToolsFactory=/atg/dynamo/service/xml/XMLToolsFactory 

XML file contains all item descriptors in the repository. and XMLToolsFactory is the out of box xml parser provided by ATG.

* The complete file.

$class=atg.adapter.gsa.GSARepository
$scope=global
repositoryName=MyNewRepository
definitionFiles=<YOUR_CONFIG_PATH>/<YOUR_XML_FILE_NAME>.xml
XMLToolsFactory=/atg/dynamo/service/xml/XMLToolsFactory
dataSource=/atg/dynamo/service/jdbc/JTDataSource
transactionManager=/atg/dynamo/transaction/TransactionManager
idGenerator=/atg/dynamo/service/IdGenerator