Solutions

Dimensions CM 14.2.0.2 - VRS Data Caching



ID:    S141604
Published:    29 February 2016
Updated:    25 August 2016

Operating System(s)

  • All Unix
  • All Windows

Product(s)

  • Dimensions CM
 

Description

This article describes the motivation and implementation of  VRS (Versioned Repository Store) data caching in Dimensions CM 14.2.0.2

Resolution

===========================================================================
Dimensions CM 14.2.0.2 - VRS Data Caching
===========================================================================

This article describes the motivation and implementation of VRS (Versioned Repository Store) data caching in Dimensions CM 14.2.0.2


Note:
VRS data caching is designed to operate automatically without any user intervention. The information in this KB article
is confidential, and is provided solely to assist in understanding and troubleshooting VRS data cache operation.

In the context of this article, a "tree" is the VRS data structure representing a project, stream or baseline.

===========================================================================
Motivation
===========================================================================

Broadly speaking, there are several classes of VRS store data.
* The name store - used to hold all the strings referenced by other data in the VRS.
* The node store - used to hold all node connectivity information for trees in the VRS.
* The revision store - used to hold all revision information for trees in the VRS.
* The node property store - used to hold all node properties for trees in the VRS.

In previous versions of Dimensions CM, DMAPPSRV processes preloaded the contents of the entire name store and contents 
of revision/node/node property stores on per-tree-version basis in their entirety for performance reasons. This
meant that each DMAPPSRV process had to fetch from the RDBMS and contain its own copy of all the VRS data needed for
command execution. If a DMAPPSRV needed to access several versions of the same tree concurrently, it usually loaded the
stores for each tree version separately leading to significant data duplication.

For large projects, this could be very expensive in terms of store load times (it could take minutes to completely 
preload a large node/revision store from the RDBMS) and resulted in DMAPPSRV private working set sizes well in excess of
200Mb per process.

By design, all stores share three properties - store data ("arcs") is versioned, immutable and append-only. New tree 
versions resulting from committed changesets result in new name/arc/value data appended to relevant stores. This makes
store data suitable for precaching on the CM server side (outside of RDBMS) and sharing between multiple DMAPPSRVs.

Thus, the goals of VRS data caching are as follows:
* Reduce overall DMAPPSRV memory usage and heap fragmentation, leading to improved CM server memory efficiency and
  increased stability
* Reduce DMAPPSRV startup time, leading to faster CM server start/stop/restart cycles
* Reduce DMAPPSRV session initialization time, leading to faster connect/post-idle-time reconnect times
* Reduce DMAPPSRV context switch time, leading to improved user experience when changing streams and projects and 
  browsing their contents in all clients, and broad improved server-side performance (most synchronization operations 
  such as UPDATE, MERGE and DELIVER, especially in incremental mode, baselining, new stream/project creation, build,
  etc)


===========================================================================
Design and configuration
===========================================================================

In Dimensions CM 14.2.0.2, the Dimensions CM server manages the VRS data cache by designating a directory on the CM server
filesystem to be used as the cache, and precaching the raw VRS store data structures on per-tree basis in files
inside that directory. Precaching is automatic and happens on-demand when a tree is accessed. It is also possible to
manually precache named projects and baselines. Old cache files that have not been used for a while are automatically
removed.

The location of the VRS data cache on the CM server's file system is defined by the DM_DBCACHE_DIR symbol in
the CM server's dm.cfg. The Dimensions CM Server installer sets it to the following value by default on Windows

DM_DBCACHE_DIR C:\ProgramData\Serena\Server\db_cache_dir\

and on Unix/Linux:

DM_DBCACHE_DIR %DM_ROOT%db_cache_dir/

The cache directory must be readable and writable by the OS user(s) used to start DMAPPSRV processes. This would be 
the pool owner if the listener pool is configured to execute in proxy mode, or the group of all users who are allowed
to login when proxy mode is not in use (when DM_ROOT/dfs/listener.dat contains the -dont_use_proxy switch).

The number of new non-cached names/arcs loaded into a private VRS store section that will trigger initial
precaching/update of an existing store is could be adjusted by setting the following symbol in the CM server's dm.cfg:

DM_DBCACHE_UPDATE_THRESHOLD 4096

'4096' is the default hardcoded value.

The cache will be periodically checked if the total size of cache files for a particular base database exceeds a certain
threshold. This is called cache cleanup. If the threshold is exceeded, the files that had not been accessed longest will
be deleted from the cache. The cache cleanup interval is specified in hours and could be adjusted by setting the
following symbol in the CM server's dm.cfg:

DM_DBCACHE_CLEANUP_INTERVAL 24

'24' is the default hardcoded value.

The cache size threshold that triggers a cache cleanup is specified in megabytes, and can be adjusted by setting the
following symbol in the CM server's dm.cfg:

DM_DBCACHE_MAXSIZE 4096

'4096' is the default hardcoded value.

As both the idle timeout and client session could be quite short, DMAPPSRVs try not to schedule cache cleanup too
frequently. To avoid this, there is a default throttle on how often a DMAPPSRV can schedule a cache cleanup. The
throttle interval is specified in seconds and could be adjusted by setting the following symbol in the CM server's
dm.cfg:

DM_DBCACHE_CLEANUP_THROTTLE 3600

'3600' is the default hardcoded value. In other words, no single DMAPPSRV will schedule a cache clean up more often than
once an hour by default.

Any time a product is deleted via DWP the DMAPPSRV will run a forced cache cleanup irrespective of the last
cache cleanup time/interval.

VRS data caching is enabled by default after installing or upgrading to Dimensions CM 14.2.0.2. It is possible to disable
it unconditionally by adding the following symbol to the CM server's dm.cfg:

DISABLE_DM_DBCACHE y

 

DMDBA command-line support

===========================================================================
A new PRECACHE command was added to DMDBA allowing the installer and CM administrators to perform initial precaching:
CM_TYPICAL> help precache
 
PRECACHE - Precache the name store, project, or baseline structure data for the current base database.
Usage: <mode> [<project-spec> | <baseline-spec> | /BULK_FILE=<file> ]
where:
   <mode>          - Specifies the type of data to be precached.
                     This parameter is mandatory and must have one of the following values:
         NAMESTORE - Precaches namestore data.
         PROJECT   - Precaches project structure data.
         BASELINE  - Precaches baseline structure data.
   <project-spec>  - Specifies a single project or stream to precache.
   <baseline-spec> - Specifies a single baseline to precache.
   <file>          - Specifies a file containing multiple project or
                     baseline specifications to precache.
   Note: The project and/or baseline specification has the same syntax as
   the value of a LIKE expression in SQL.
  
  
Usage examples:
DMDBA cm_typical/cm_typical@dim14
CM_TYPICAL> PRECACHE NAMESTORE
CM_TYPICAL> PRECACHE PROJECT %
CM_TYPICAL> PRECACHE BASELINE QLARIUS:%

 

 

Applies To

Dimensions CM 14.2.0.2

Rate this Solution

Find Answers

Type a question or describe what you are looking for below

My Recent Searches

Welcome kb sso

Additional Assistance

  • Submit a Case Online
  • FAQs