|
|
|
In This Section: |
SmartReporter is a Check Point reporting product that extracts and consolidates network security events to create concise predefined or custom-created reports. You can generate reports for network, Firewall, endpoint status, Threat Prevention, and other activities.
SmartReporter uses a Consolidation Policy to extract data from log files. It consolidates equivalent log types and adds them to the SmartReporter Database. This database lets SmartReporter quickly and efficiently create reports.
The SmartReporter Policy is a collection of Consolidation Rules that control which log information to extract and consolidate. These rules filter extraneous logs and consolidates the most important (blocked connections, alerts, web activity, and so on). The Log Consolidator reads the Consolidation Rules sequentially and processes each log according to the first Rule that it matches.
When a log matches a Consolidation Rule, it is either ignored or stored. If it is ignored, no record of this log is saved in the SmartReporter database and the data is not available for report generation. If it is stored, it is either saved as is (so all log fields can later be represented in reports), or consolidated to the level specified by the Rule.
Consolidation is done on two levels: the interval at which the log was created and the log fields whose original values should be retained. When several logs matching a specific Rule are recorded within a predefined interval, the values of their relevant fields are saved "as is", while the values of their irrelevant fields are merged (for example, "consolidated") together.
In DHCP address mapping is used. Assuming the DNS knows how to resolve dynamic addresses, the information you see in the report reflects the correct resolving results for the time the reported log events have been processed by the SmartDashboard Log Consolidator and inserted into the database.
Because of the dynamic nature of DHCP address distribution, there is no guarantee that consolidation of old log files will produce correct address name resolving.
When DHCP is in use, consolidating log files close to the time of their creation will improve address-resolving accuracy.
DBsync enables SmartReporter to synchronize data stored in different parts of the network. After SIC is established, DBsync connects to the management server to retrieve all the objects. After the initial synchronization, it gets updates whenever an object is saved. In distributed information systems DBsync provides one-way synchronization of data between the Security Management Servers object database and the SmartReporter computer, and supports configuration and administration of distributed systems.
With DBsync, initial synchronization is established between the SmartReporter computer and the Management server computer (for example, Security Management Server or Multi-Domain Server). In a Multi-Domain environment, you can choose which domains to synchronize in the SmartReporter client, in the Domain Activation menu. If the initial synchronization is not complete the administrator will receive a warning informing him that the GUI will open in read-only mode. Once initial synchronization is complete SmartReporter will open in Read/Write mode.
As a result of DBsync, whenever an object is saved (that is, a new object is created or an existing object is changed) on a Management computer the object is automatically synchronized in SmartEvent.
|
Note - When working in Multi-Domain Security Management mode you must select Domains that will initiate synchronization with the Domain Management Server of the selected Domain (Tools > Domain Activation). |
Synchronization can take time up to 30 minutes, although this is usually the time needed for a very large database.
The SmartReporter client offers a wide selection of predefined reports for both Standard and Express reporting, designed to cover the most common network queries from a variety of perspectives.
The Log Consolidation process results in a database of the most useful, relevant records, known as the SmartReporter Database. The information is consolidated to an optimal level, balancing the need for data availability with the need for fast and efficient report generation.
Reports are generated based on a single database table, specified in the Reports view > Standard Reports > Input tab. By default, all consolidated records are saved to the CONNECTIONS table and all reports use it as their data source. However, each time you create a new consolidation session, you have the option of storing records in a different table.
Dividing the consolidated records between different tables allows you to set the SmartReporter client to use the table most relevant to your query, thereby improving SmartReporter server performance. In addition, dividing records between tables facilitates managing the SmartReporter Database: you can delete outdated tables, export tables you are not currently using to a location outside of the SmartReporter Database and import them back when you need them.
Express Reports are based on data collected by SMTP-like system counters and SmartView Monitor history files, instead of logs. Because Express Reports show historical data, you cannot filter the results, but the reports are generated faster.
Express Reports are supported by one Client, the SmartReporter. To configure your system to generate Express Reports, see Express Reports Configuration.
Each report consists of a collection of sub-topics known as sections, which cover various aspects of the report. For example, the User Activity report consists of sections such as User Activity by Date, Top Users and Top Services for User Related Traffic.
You can easily customize the report that is closest to your needs (by changing its date range, filters etc.) to provide the desired information. Changing the filters of a predefined report constitutes a change in the nature of the report and the report must therefore be saved in a different location or under a different name. You can save the customized report under a different name in the report subject dedicated to user-defined reports, My Reports.
SmartReporter default options have been designed to address the most common reporting needs. To maximize product benefits, it is recommended that you adapt it to your specific profile. This section describes the considerations you should take into account before starting to use SmartReporter.
In a standalone deployment, all SmartReporter server components (the Log Consolidator Engine, the SmartReporter Database and the SmartReporter server) are installed on the Security Management Server. In a distributed deployment, the SmartReporter server components and the Security Management Server are installed on two different machines. They communicate through standard Check Point protocols such as LEA and CPMI.
In a standalone deployment, you can use one server for all of the management components. In a distributed deployment, the SmartReporter performance is significantly improved.
In a standalone deployment, you can install SmartReporter on a Security Management Server of the same version. In a distributed deployment, you can install SmartReporter on a Domain Log Server and manage it with a Security Management Server of any supported version.
All SmartReporter operations are based on the logs. Make sure that your SmartReporter Policy creates logs all events that are important for your organization.
Make sure that your logs show your network activity accurately. If only some rules are configured to create logs for matching events, those events will show will show in your reports and the others are ignored. For example, if only blocked connections generate logs, the reports will show that 100% of your activity in your network is blocked connections.
On the other hand, creating logs for too many different activities can create a very large log file, which slows the log consolidation operation and shows too many events.
Reports are a direct reflection of the records stored in the SmartReporter Database. To generate detailed, wide-ranging and accurate reports, the corresponding data must be available in the database. You must configure the database settings to make sure the database does not exceed the available space.
Carefully consider which type of logs you store and how much you consolidate them.
A report is generated based on a single table. If you save all consolidated records to the same table, all the data is readily accessible and you are saved the trouble of moving records between tables and selecting the appropriate source table for each report you wish to generate.
Dividing the records between different tables reduces the report generation time and allows you to maintain a useful database size by exporting tables you are not currently using to an external location.
SmartReporter supports Security Management Server High Availability.
In High Availability, Security Management Servers must all be of the same operating system (for instance, all Windows NT), and have to be of the same version. The existence of the standby Security Management Server allows for crucial backups to be in place:
In a High Availability deployment the first installed Security Management Server is specified as the Primary Security Management Server. This is a regular Security Management Server used by the system administrator to manage the Security Policy. When any subsequent Security Management Server is installed, these must be specified as Secondary Security Management Servers. Once the Secondary Security Management Server has been installed and manually synchronized, the distinctions between Primary versus Secondary is no longer significant. These servers are now referred to according to their role in the Management High Availability scenario as Active or Standby, where any Security Management Server can function as the active Security Management Server.
When changes are made to report definitions (including report schedules), consolidation sessions and their settings, automatic maintenance configuration and report configuration, the information is stored in the active Security Management Server and will be synchronized to the secondary Security Management Server when a user synchronizes the Security Management Servers.
The report generation results are not synchronized between Security Management Servers. For instance, when SmartReporter generates a report connected to Security Management Server A, a record of its generation will be stored in Security Management Server A. When SmartReporter generates a report connected to Security Management Server B, a record of its generation will be stored in the Security Management Server B. The Activity Log in Security Management Server A will not be visible in Security Management Server B and vice versa. However, even though the Activity Log in the inactive Security Management Server A is not visible, it is still possible to connect to the inactive Security Management Server A in read-only mode to access the report generations that are not visible in Security Management Server B.
When a report is very detailed, it may become difficult to sort out the most significant results and understand it. To achieve the optimal balance between getting the right level of detail in your reports, closely examine report date range, filters (source, destination, service etc.) and filter values, and adjust them to pinpoint details.
By default, specific sections are included in the report generation and sections that require a great deal of resources (that is, report generation time and the report size) are not selected. However, you can generate any sections in the list provided by checking them in the Content tab associated with the selected report in the Reports > Definitions view.
The Schedule feature allows you to set both delayed and periodic report generations.
If you wish to produce a detailed and lengthy report, you should consider postponing its generation and scheduling it so that it does not run at time of peak log creation activity since such a report generation might slow down your system.
In addition, it is useful to identify the reports you require on a regular basis (for example, a daily alerts report or a monthly user activity report) and schedule their periodic generations.
Reports are based on records of the most commonly required filters, such as Source and Destination. Specifying the appropriate filter settings is the key to extracting the information you are looking for.
For each filter you choose, specify the values, such as network objects or services, to be matched out of all values available for that filter. The available values are taken from the Security Management Server and are refreshed on a regular basis. If you cannot see a value you have added through SmartDashboard in the available values list, refresh the list by selecting a different filter and then return to the previous one.
The SmartReporter client also allows you to include additional objects by manually adding them to the matched values list.
Filters and their values can be specified for all sections of a report using the Filter tab, or for individual sections by editing the section from the Content tab. Filters for individual section set in the Content tab will override conflicting filters set for all sections using the Filter tab.
All report results are displayed on your screen and saved to the SmartReporter server.
By default, the report is saved in HTML output in an index.htm file; and in CSV (Comma Separated Values) format in a tables.csv file. The HTML file includes descriptions and graphs, but the CSV file contains only the report table units, without a table of contents, descriptions or graphs. The tables.csv is provided in order to conveniently import tables into applications like Excel.
File Format |
HTML |
CSV |
|---|---|---|
File Name |
index.htm |
tables.csv |
Includes |
Table of contents, tables, descriptions, graphs. |
Data only. Cell values separated by commas. Rows and tables separated by lines. |
Before generating a report, determine whether you want it to be saved or sent to additional or different targets. For example, when you generate a user activity-related report, you may wish to make it available to all managers in your organization by sending them the output via email or by placing it on your intranet.
This release can use one of these SQL databases:
You do database management operations in these ways:
To see which SQL database is installed, run:grep DefaultDatabase $CPDIR/registry/HKLM_registry.data
If the command returns the string PostgreSQL, the database is PostgreSQL. If the command returns another result, the database is MySQL.
You can configure the MySQL database to improve performance. We also recommend that, if possible, you put the main database files on a different physical disk than the log files and other temporary files. This lets the report generator run faster and makes sure that there is always sufficient disk space for the database, logs and temporary files.
Configuration file for MySQL:
On SecurePlatform or Gaia platforms, the database configuration file is $RTDIR/Database/conf/my.cnf. On Windows platforms it is %RTDIR%\Database\conf\my.ini.
Configuration file for postgrSQL:
On SecurePlatform or Gaia platforms, the database configuration file is $RTDIR/events_db/data/postgresql.conf. On Windows platforms it is %RTDIR%\events_db\data\postgresql.conf.
You can change the SmartReporter database settings by modifying the my.cnf file, located in the $RTDIR/Database/conf directory (in Windows: my.ini). Run the UpdateMySQLConfig application. Note that before running this application you must stop all SmartReporter services: run evstop -reporter.
When you run the UpdateMySQLConfig application, it creates a backup of the database configuration file.
There are a number of factors that can improve performance of the SmartReporter database. Most of these factors can be changed with the UpdateMySQLConfig utility.
The default database file is ibdata1. If this file needs to be moved to a new absolute directory (for example, d:/Database/data), verify that the directory exists and run:
UpdateMySQLConfig -M -src=ibdata1 -dst="d:/Database/data/ibdata1"
If you want to remove an absolute directory (for example, d:/Database/data2 to d:/Database/data2), verify that the directory exists and run the following:
UpdateMySQLConfig -M -src="d:/Database/data/ibdata1" -dst="d:/Database/data2/ibdata1"
Important - You cannot lower the maximum size of the database. Doing so could result in database failure.
Changing the Database Data Directory
The location of the database data files is specified in the MySQL configuration file my.ini (Windows) or my.cnf (all other platforms).
- datadir=
- innodb_data_file_path=
The directories indicated by these entries are the directories and subdirectories that should be copied to the new location. The following example shows how these directories show in the MySQL configuration file.
[mysqld]
datadir="C:/Program Files/CheckPoint/EventiaSuite/R77/ReportingServer/Database/data"
innodb_log_group_home_dir="C:/Program Files/CheckPoint/EventiaSuite/R77/ReportingServer/Database/log"
innodb_data_file_path = ibdata1:10M:autoextend:max:40G
The entry innodb_data_file_path, records database files that were added or moved to absolute locations. Make sure that these recorded database files are copied to a new location so that they are not forgotten.
Make sure that the paths are written in Unix format, with forward (/) slashes between directories.
The UpdateMySQLConfig Options table contains the usage of the UpdateMySQLConfig application.
Syntax
UpdateMySQLConfig[-A -f=string -s=number -auto[=true|=false] [ -m=number ] ][-R=number ][-M -src=string -dst=string ][-T=string ][-L=string ][-h ]Parameters
Parameter |
Sub-parameter |
Description |
|
|
Add a new data file to the database. |
|
||
|
||
|
||
|
|
Sets the level of database RAM usage. |
|
|
Moves a database file to a new location. |
|
||
|
|
Changes the path to MySQL temporary directory |
|
|
Changes the path to MySQL log directory and copies log files to the new location. |
|
|
Displays this help message. |
The Log Consolidator process continuously adds new records into the database as they are generated from the Security Gateway. Eventually, the space allocated for the database will fill up. Typically, users can manually archive or delete older, less pertinent records from the database to provide space for the newest records. Automatic Maintenance performs this process automatically. With Automatic Maintenance, the user selects a maintenance operation (whether it is deleting records or archiving them to an external file) and specifies high and low watermarks to trigger when Automatic Maintenance should occur.
The High Watermark value represents the percentage of space that can occupy the database and/or the age of database records (that is, how many days old the records are). When the database occupies too much space or the records are older than the specified age, then the conditions are right to trigger an Automatic Maintenance operation. The High Watermark values are checked once a day and if the percentage of space or the age of the database records is higher than the assigned values, the Automatic Maintenance operation is triggered.
The Automatic Maintenance operation will delete records from the database until it reaches the Low Watermark. For example, if you specify that the High Watermark is 80% and the Low Watermark is 70% then the operation will begin to delete the oldest records when the occupied space is over 80%.
Typically, it is recommended that 80% would be the High Watermark to avoid reaching 100% capacity in certain cases.
In addition, it is possible to specify which database tables will participate in Automatic Maintenance. Since some of the tables are created for special purposes (for example, a table created from an external log file), Automatic Maintenance should not be performed on them.
When deletion of records occurs during automatic maintenance, you may see that the database size grows at first. This is normal behavior since the database needs to keep duplicate information in case of a server failure. The database will recover the disk space allocated for logs for about an hour after the maintenance operation is complete.
The SmartReporter Database system consists of a set of files that can be copied, compressed or backed up like any other file. Backup files require the same disk space as the original files. It is highly recommended to save backup copies of the SmartReporter Database files, which can later be used to recover from an unexpected database corruption. Proceed as follows:
evstop -reporter.rmdsstart.To change the SmartReporter database size, run this command:
cpprod_util CPPROD_SetValue "Reporting Module" "PostgresDatabaseSize" 4 <SIZE of database in KB> 1
Example:
cpprod_util CPPROD_SetValue "Reporting Module" "PostgresDatabaseSize" 4 200000 1
This example sets the database size to 200 MB. When the database size is greater than this value, the oldest log entries are deleted to make room for new entries.
If the free space on the database partition is less than the defined threshold, the system deletes the oldest log entries. The default threshold is 15% of the partition or disk size. You can change the threshold using the command line.
To change the minimum free space threshold:
change_configuration.txt. The location of this file is not important.modify abacus_db_mgmt db_mgmt_properties free_size_threshold <New %>
<New %> - New threshold expressed as a percentage of the disk or partition.
dbedit -local -f /path_to/change_configuration.txt
abacus_db_mgmt::db_mgmt_properties was not updated' prompt, enter y.This message shows if the merge is successful:
abacus_db_mgmt::db_mgmt_properties Updated Successfully
To open a shell directly to the SmartReporter database:
Run $CPDIR/database/postgresql/bin/psql -U cp_postgres -p 18272 rt_database.
This location is the same as the SmartEvent database ($RTDIR/events_db/)
To back up the SmartReporter database:
evstop. $RTDIR/bin (Gaia, SecurePlatform or Linux)%RTDIR%\bin (Windows)eva_db_backup.csh -filename <backup file_name> -database database EvrDb.evstart. To restore the SmartReporter database:
evstop. eva_db_restore –filename <backupfilename> -database EvrDb
evstart.
|
Note - When you restore a PostgreSQL database, the existing database and its contents are deleted. The restore procedure does not add the backup database contents to the existing database. |
This procedure moves the SmartReporter PostgreSQL database to a different disk location. By default the SmartReporter database is installed at:
$RTDIR/events_db/data (Gaia, SecurePlatform, Linux)events_db/data (Windows)To find the current database location:
$CPDIR/registry/HKLM_registry.data (Gaia, SecurePlatform and Linux)
PgDataPath attribute. $CPDIR/registry/HKLM_registry.data
PgDataPath attribute.To Move the SmartReporter database to a different location:
cpstop. $RTDIR/events_db (Gaia, SecurePlatform, Linux).
%RTDIR%\events_db/data (Windows).
mv data <new_path>.$CPDIR/registry/HKLM_registry.data (Gaia, SecurePlatform, Linux.
PgDataPath attribute. PgDataPath attribute to the new path. cpstart. Note: This procedure also moves the SmartEvent database to this new path.
You can delete an existing SmartReporter database together with all of its content. This procedure also creates a new, empty database.
To delete a database and create a new database:
evstop. $CPDIR/database/postgresql/bin/psql -U cp_postgres -p 18272 postgres -c 'DROP DATABASE rt_databasepprod_util CPPROD_SetValue "Reporting Module" DatabaseCreated 1 "No" 1evstart.This procedure can take a long time to complete.