High-Performance Knowledge Base support for Monitoring, Analysis, and Interpretation Tasks

The Monitoring, Analysis, and Interpretation Tool Arsenal (MAITA) project was
initiated to develop knowledge based tools for constructing monitoring systems
aimed at reducing the costs-in time, effort and expertise of constructing a
monitoring system at the start and of modifying an operational system of monitors
to address special and temporary concerns. The MAITA architecture thus provides
for a rich library of monitoring systems and mechanisms enabling easy composition,
modification, testing and abstraction of these library elements.The MAITA system
was developed concurrently with explorations of applications to knowledge-based
battlefield movement and computer security monitoring. In the initial battlefield
application, MAITA monitoring processes successfully detected convoys and other
battlefield movements in simulated airborne doppler radar data. The later computer
security application was in preliminary stages at the end of the grant period but
has subsequently detected intrusions and other events in simulated network data.
The MAITA system provides tools both for constructing intelligent monitoring networks
that exploit domain knowledge to shift and correlate source-level signals, and for
rapidly modifying running networks to address specialized or temporary concerns. The
remainder of this overview outlines the motivations for the overall system organization.

Distributed monitoring

The central concept in the MATTA architecture is that of a network of distributed monitoring
processes. The metaphor we use for thinking of the operation of these monitoring networks is that
of electrical networks, in which we "wire together" various components and network fragments by
connecting their terminals together. In the computational context, individual monitoring processes
take the place of electrical components, and transmitting streams of reports takes the place of
electrical conduction. The set of monitoring processes form the nodes of the network, and the
communication paths form the edges or links of the network. Each process in the network may
have a number of "terminals", each of which receives or emits streams of reports. The network
may exhibit a hierarchical structure, as some monitoring processes may consist of a subnetwork of

Stable monitoring

The distributed processes may degenerate into chaotic interference without some means for
structuring the interactions. To provide this structure, MATTA provides a "monitor of monitors" or
"MOM" to construct. maintain, inspect, and modify the monitoring network and its operation.
We achieve a degree ,uniformity in the control process by organizing MOMs as special types of
monitoring processes.The MOM is designed to provide' for resilient and perisistent networks of
monitoring processes. Toward this end, the command and control system monitors all the other
monitoring processes, correcting and restarting them as needed. The control system itself is
monitored by a subsidiary monitor which corrects and restarts the control system as needed.
The architecture employs a persistent database to aid in providing this level of stability, and
monitors the functioning of the database system as well. The control system also works to ensure
the accuracy of the database records, both by updating them as changes are made and by checking
them as needed.

Secure monitoring

The architecture provides for a fairly standard Unix like scheme of users, groups, passwords, and
permissions, specialized for the distinctive classes of operations performed on monitoring networks
and their elements. The architecture also is designed to provide a minimal target for would-be
attackers by establishing most of its data communications through ephemeral listeners operating out
of randomly assigned ports. This leaves the only entry points of the system knowable in advance to be
the main starting address of the system. Even this can be varied at the time of system startup. and
across independent MATTA systems. Future improvements to the system may enable changing of this
main address during systnem operation as well, allowing a form of "frequency hopping".
The initial design of the MATTA system presumes that the most security issues involving the
control system must involve Inechanisms external to MALTA that provide the operating context of
the monitoring process.

Open monitoring

The architecture is designed to provide an open platform for system development and interconnection.
Command operations are transmitted using hypertext transport protocol (HTTP), allowing
for basic system operation from any web browser, using commands entered by hand or through
multiple specialized web pages or applets. Such web-based control minimizes requirements for
installing specialized software on local machines. Supporting this open operation further, we provide
reference implementations of the MAITA specific communications mechanisms, in the form of
Java and Common Lisp classes that provide monitoring process wrappers for use in legacy systems
written in these languages. Data are transmitted by an expandable set of common protocols, permitting
direct interconnection with many legacy and separately developed systems. The design permits
information to flow through the network by several different protocols, including socket based
ASCII character streams, HTTP (Hypertext Transport Protocol, used by the World Wide Web), SMTP
(the Simple Mail Transport Protocol, used by email systems), Java RMI (Java Remote Method Invocation),
ODBC (Open Database Connectivity), and OKBC (Open Knowledge Base Connectivity, a protocol for
transmitting logical and frame-structured knowledge to and from knowledge bases). The system developer
or user chooses the protocol appropriate to the volume, regularity, and type of the information being
transmitted. Regular and high-frequency transmissions typically go through persistent stream, ODBC, or
OKBC connections. Intermittent and low-frequency transmissions probably go on temporary HTTP, SMTP, Java
RMI, ODB-C, or OKBC connections. Records of information transmitted to input or from output terminals
are structured: in protocol dependent formats.

Intelligent monitoring

The first role is that of monitoring processes which explicitly reason in the course of their
analysis. MAITA supports these by offering an ontology of monitoring concepts and a knowledge
base of monitoring methods. We annotate the structure of information flow with knowledge-level
descriptors, distinguishing the reports being transmitted and received from the computational
representations (packets) of these reports, and distinguishing these computational representations
from the protocol-specific encodings used for transmission.

The second role is that of monitoring networks, in which the structure of the network explicitly
reflects knowledge about the conditions being monitored. This network structure should identify
the conditions of interest and the dependencies among them. For example, the structure of a
monitoring network should revolve around the conditions on which attention should be focussed,
and provide checking of expectations (both positive and negative) related to these conditions.

The third role is that of alerting models, in which knowledge about the likelyhood of different
classes of alerts or reports, time and other costs of transmission, and utility to different recipients
or recipient classes is used to make rational choices about who to tell what, and when and how.
Evolutionary monitoring

Sensible engineering design calls for components that may be reused or adapted in subsequent
designs. The MAITA libraries provide means for abstracting, recording, and sharing monitoring
networks developed for one purpose with developer of monitors for other purposes. These libraries
aim to provide a broad and deep base of abstract, and concrete monitoring methods, event,
descriptions, and alerting models.

, , , , , , , ,

  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: