Infrastructure Documentation

General Information
Configuration
Communications
Events
Alarms	Calling a routine at an absolute or relative time.
Threads
Perl Support
Daemons/Servers
Tracing & Error	Program tracing and reporting.
Tools / Program Control	Real time interaction with running systems or individual programs.
Utilities
Function References	Function refs for all public functions broken down by layer.
Document HowTo (XML)	How to create layer documents using the infrastructure's xml to html and indexing system.
Download Inf.tgz

• General
  The infrastructure is a set of cohesive software tools (for 'C' programmers) and programs which form the basis for Linux (UNIX) client server systems.   The infrastructure is also called middleware.   Partitioned into layers, each responsible for specific groups of related operations.   As of 01/10/2007 ther are over 37,000 lines of code in the infrastructure.
  
  The source code currently compiles and runs on Linux, but in the past has been run on HPUX, AIX, FTX, AT&T Unix, and Solaris, so most of the nitty gritty stuff should be easy to get running on other platforms.   It is also compiled using gcc (forget about the HPUX ansi or bundled compiler).
  
  The heart of the infrastructur's ability to handle high volumes of application transactions while giving applications independence from one another is the Message Object and the Orb (Object Request broker).   Message Objects may optionally be sent over encrypted paths by simply changing the configuration file.
  
  Each process requires a configuration file which relates it's logical name to it's executable name and provides other configuration information directly into program variables.   The same executable may be run numerrous times, under sysgo, with different logical names.  
  
  Each process has an input queue for communication using the default message type 'EVTDRVR'.   Other queues may be established, listening on various machine interfaces and ports, for in-bound communication.   The type of a message may be changed and message types may be assigned to event call-backs to provide special or faster handling of some messages.   Each queue definition must contain at least 4 elements: Protocol, logical queue name, location (hostname), and physical (port).   Two additioal elements may be added: Encryption Type (blowfish or AES) and an Encryption Key.  
  
  A single wait list must be used for an entire process, this means the wait list must be managed so that all events to be waited upon are constantly reflected by that list.   The wait function may be called from multiple places in the code but they must share the wait list.
  
  
• Conventions:
  At one time or another, the infrastructure has run on AT&T Unix, FTX, AIX, HPUX, Solaris, Sequent Unix, and Linux.   but recently it has become more and more Linux oriented and should probably be compiled using gcc.  
  In the case where a macro is wrapped around a function call the actual function definition starts with an underscore and the macro has the same name without the underscore.
  Global Variables begin with capital letters. Very few global variables are exposed beyond the module.
  Most Macros (static definitions) are all uppercase.
  All typedefined macros end in "_t"
  The public functions within a layer have a two or three lower case letter prefix which identifies the layer. Functions beginning with an underscore are referenced from macros.
  
  How to make inf:
  Get the inf.tgz and unzip it in a directory called inf, then type:
  make unzip
  make first
  Your will be asked for the root password.
  You will need to be sure that /etc/ld.so.conf contains the line: /usr/local/lib then type:   make install
  You will be asked for the root password, enter it and you will have a development version of the infrastructure installed on your computer.   You will also have a copy of these web pages starting at the (the directory you unzipped the inf.tgz into)/docs/index.html.   Just open that index.html file in firefox.   You can now create an distribution (installation) package by typing:
  make dist
  This creates the package infDist.tgz, which may be moved to another computer in it's /usr/local directory and untarred.   When you untar the dist package in /usr/local, it installs a number of executabtls in bin/, some libraries in lib/ and some configuration files in etc/.   You can make other targets from the development install like "tar" (creates inf.tgz).  
  
  
• Configuration
  is accomplished via a flat text file which may configure a single process or an entire "Engine" of processes.
  Each process must have a configuration file to define it's logical name and specifics about the I/O.   This file may also be used to configure the application layer and infrastructure layers (see config.rtf).   An engine (system of processes) may be configured from a single configuration file.
  
  You may make program variables or initialization call-backs directly accessable from the configuration file.   The program variables are related to KEYWORDs in the config file, using a configuration table in you module's global variables, then registering the table using the CFG_REG macro prior to calling infInit().   The macros CFG_bGN CFG_INT, CFG_STR, CFG_FLG, CFG_FUN, and CFG_END, are used to build this configuration table.   The macro CFG_FUN defines a KEYWORD as a specific call-back in the application code which will be passed the line from the config file whcn the KEYWORD is encountered.   Please see examples in inf/include/cfg.h and in asTempl.c.
  
  Process configuration and initialization may be performed in two phases:
  An application program initializes by registering it's config, option, control, and service tables then calling infInit() which actually reads the configuration file.
  In the case the application needs to do more initialization after the confiuration variables have been read from the config file, the application programmer may register a post-config call-back, EVTINI2 which is fired after reading the config file.
  In the future additional phases may be added to support multi-process (engine wide) initialization interaction (i.e. Servers should be up and initialized before the clients begin making requests).
  
  
• Communications
  There are two levels of communication supported by the infrastructure, io and msg.   All packets travel across bi-directional permanent virtual circuits in non-blocking mode.   Low Level I/O (io) for basic, very effecient packet communication, uses a character pointer pointing directly to the message storage area.   This layer also does the encryption.
  High level I/O (msg) supports, complex opaque message objects, multi-packet (very long fields), packet router, and uses integer handles to reference messages.  
  
  The law of message conservation When a message is created or received it will exist until it is sent or the function call to free it is made.   The action of "sending" a message frees it so you must duplicate it if you need more information from the original.   You can not use the "free()" call directly on a message reference, you must instead, use the creating layer's free function (ioFree( mp ) or msgFree( hndl )).   Messages are created in three ways: directly created, duplicated from another message, or received.
  
  The basic I/O layer is very efficient and is used to transport all higher level layers of communication.   The function prefix for this layer is "io".   The primary reference is a character pointer which may be used to directly access the data.   If an mo is placed in the message area mo function calls must be used to access it using the original I/O message pointer.  
  
  The messaging (msg) layer moves message objects (mo) containing multiple fields grouped into records of any size between two processes.   The function prefix for this layer is "msg".   A message object field has a name and value.   The internal representation is determined when the field is inserted into the mo and is either integer, string, or binary.   A field stored as a string may be retrieved as an integer and vice versa.   There are some limitations on the retrieval of binary fields.   The first record in an mo is always record number one.   When extremely large messages are sent, multiple packets are used.   The fields and records in a multi-packet message will be kept intact during transmission.   The primary reference is an integer handle which represents the opaque object.
  
  
• Events
  Heavy use of events provide numerous ways to "tap in" to internal facilities as well as send events between processs even across the network.   There are two kinds of events, please click HERE for more info.
  
  
• Threads
  Posix Threads (pthreads) are primarily supported in the msg layer and are usable as clients only.   That is, a server may use pthreads to make multiple requests of other servers or just to do parallel processing but the Orb sees the process as a single server.  
  
  
• Tracing
  Allows the placement of trace and error statements in 'C' or perl code at strategic places assigning priority by system layer and level which are set at run-time.   The calling function name is inserted at the beginning of the message in both trc() and trcErr() calls.   This may be inhibited by placing a dot '.' character at the beginning of the format string.   Trace and error logs are rotated automatically at midnight.
  Trace statements may be placed anywhere in the thread of execution.   Each trace statement contains a control which is broken into two parts, the layer mask and the level.   The layer masks are used to select which layer's trace statements will be enabled and the level controls the amount of granularity.   It is customary to use a higher level to get more trace output.   Each process has global variables which contain the current settings for the layer and level of that process.   These variables may be set at initialization time from the configuration file, the command line, or ab environment variable.   The process' mask and level variables may be changed at run time using program calls or the control tool (ctl).   The program can make a function call to determine whether tracing will be done so that specific trace code may be bypassed when in production mode (tracing not required).
  
  The trace information destination is one of three places: first priority is to a trace daemon (trcd), second priority is to a file on disk, and third priority is to stderr/stdout.   Trace data is always stored on disk in a binary file which must be read using the 'extr' command.   If the file is created by a trace daemon, it is called "trcLog", and if it is created by a local process it is called "localProcess.trc" (where the process name is the logical process name from it's config file).   A process will create it's "ProcName.trc" file in it's current directory unless "LOGDIR" is encountered in it's config file wich changes the trace file's location.   The trace daemon (trcd) aggregates the output of multiple processes.   Each entry in the trace file contains the time (to the millisecond) the logical name of the originating process and the trace text.
  Message objects may also be traced when the "msg" layer is used.   Messages may be traced a) when it arrives at a specific process, b) when it leaves (is sent from) a specific process, or c) a specific process will mark it's messages when they are sent so that every process which touches the message will trace it.   These functions are implemented and controlled as if they were layers in the system.
  
  
• Demons / Servers
  The ORb (Object Request broker) assigns message objects to servers using a service type number contained withing the MO.   The load is balanced across all servers of a single type.   The ORb does reliable queueing, marshaling, poison pill detection, request reassignment in case of server crash, in addition to load balancing.   ORb can be configured to immediately acknowledge a request from a client (fire and forget) or wait until the service is complete (default).   ORb keeps complete statistics on each service type as to queue time and service time, arrival rates, and total counts.   A document on ORb configuration can be found here.
  
  The Trace daemon (trc) supports tracelog and errorlog writing.   Trace messages from all processes in the system are interdigitated and sorted by timestamp into a single logfile so that events may be viewed in exact time relation to each other.   The logfile is rotated each night at midnight or it can be rotated using the infCtl (Infrastructure Control) tool.
  
  Program Launcher is used to run command line programs, including perl, capturing the stdout and stderr, then returning this ascii text to the requestor.
  
  Message Router is used to route requests to ORbs on remote machines, usually to a launcher.
  
  Cache Daemons CryptoCache and CfgCache are used to hold and restore configuration information for running programs.   These daemons are consulted with a single call to the caGetCfg() or caGerPwds().   This call is usually made at the beginning of a service cycle and since the cached data can be changed anytime, this allows dynamic reconfiguration of all daemons.
  
  
• Tools & Program Control
  
  
• Cache Daemons
  These tools are primarily to be used from the command line, some are also suitable to be run from scripts.
  
  infman
  This is the infrastructure man page tool. When you install the infrastructure on a Linux machine (and some other Unixes) man information (in xml) is saved in /usr/local/man/inf. If you execute infman with no arguments it lists the infrastructure modules. If you pass a module name as an arguemnt, infman lists all prototypes in that module, and if you pass modulename.functionname it lists the function prototype and description.
  
  sysgo   configName
  Performs initiation, control and monitoring of a group of processes, or engine.   Like all other processes, sysgo requires a configuration file, but this file is used to create configuration files for multiple sub-processes.   Each sub process runs in it's own sub directory below the "INFHOME" directory and has it's stdout and stderr redirected to files of the same name, int that sub-directory.   If sysgo is run from the inittab, you must use the keyworkd FOREGROUND in the PROC stanza.
  
  sysstop  configName
  A tool used to shutdown infrastructure engines or all processes running under a sysgo.   May be run from command line or script.
  
  killproc   configName   processLogicalName
  A tool used to kill a single process running under sysgo.   May be run from command line or script.
  
  extr   [options]
  Extract from trace or error logs.   Trace and error log files are assumed to be in the current directory, unless a path is included.   The options below, determine what records will be extracted and displayed.
  With no options, extract from current trace log.
       [e] From current error log.
       [xxx.xxx] From specified file (\"trcLog.2\").
       [-b hh:mm[:ss]] begining at time (inclusive).
       [-e hh:mm[:ss]] Ending at time (inclusive).
       [-p processName] Only named process' trace records.
       [-v] Verbose.
       [-m] Eplicate Message Objects.
       [-t x,y] Extract only messages of type x and y.
  
  infCtl   configName
  Infrastructure control "infCtl" tool may be used to directly control a process or system of processes.   InfCtl allows access to the "control" function built into each daemon.   These "control" functions display stistics, configuration variable content, and tracing controls.   Most of these can also be set by infCtl.   Application functions can be built for direct interaction with the infCtl tool.  
  
  infRmt   configName
  Infrastructure Remote "infRmt" tool may be used to directly control a remote group of system of processes.   Works like InfCtl but has a different configuration file and an extra menu to select which remote system to contact.  
  
  
• Utilities
  These packages are separated into Utility and Infrastructure libraries so that they may be used in different circumstances.   The utility functions may be called from any single stand alone or infrastructure process.   The Infrastructure functions support multi-process systems and facilitate communication between and control of these processes.   Each library contains layers dedicated to a specific feature.
  
  Infrastructure Layers

  inf	Infrastructure top layer
  appSrv	Application server template and support.
  sys	Process control and configuration.
  stt	Statistics accumulation and display.
  msg	Combines mo and I/O, adds trace display of messages and mult-packeting
  ctl	Real-Time system and process control
  trc	Tracing of program flow and display of communication contents
  IO	Input Output (Socket) using non-blocking, permanent virtual circuits.
  alrm	Perform tasks at specific or relative times.
  cfg	Process and layer configuration.
  Utility Layers
  mo	Message Object: Opaque data storage for communication.
  str	String processing
  charIo	Character I/O
  avl	Sorting and access (primitive)
  txt	Text File handling
  opt	Command line options
  xml	Converts documents into html.