Processing multiple files

Using ftp

Summarization utility

Historical files

Using Windows Script Host (WSH) scripts

At the end of each collection cycle, the Collection service closes the current collection file to make it available for processing. The end of a cycle normally is used to initiate processing of the collection files in SAS or MXG. Cycle end processing normally consists of scheduling a process to consolidate Performance SeNTry data files at a central location. This section discusses automating cycle end processing.

The Collection service stores data collection logs in the \Data subdirectory under the NTSMF24 program directory by default. The location of the data directory can also be changed using the SeNTry Administration program by modifying the Directory where SMF files are stored start-up parameter.

Hierarchical file management sets up Current, Previous and Archive Folders under the \Data directory and manages collection files according to the criteria you specify. The hierarchical file management option is specifically designed to simplify your cycle end procedures. By default, old collection files are moved immediately from the Current Folder to the Previous Folder. After the number of days you specify, they are then migrated to the Archive Folder. Files are kept in the Archive Folder for an additional period of time and then deleted automatically according to your specifications. Because all automatic file management is performed prior to issuing the Cycle End Command, the cycle end procedure you execute can be as simple as a COPY *.* command pointing to the Previous Folder.

Unique collection data log file names are automatically generated of the form

systemname.yyyymmddhhmmCustomQualifier.smf

where yyyymmddhhmm is a date/time stamp identifying when the file was created. During the collection cycle, the Collection service can lock the current data file so that no other application can access it. Or the current data file can be Shared so that it can be processed by other Windows NT/2000 programs or commands while collection is occurring and the file is open. The collection file is closed at Cycle End and is available for processing.

The Collection service automatically launches the Cycle End Command you specify after all file management actions have been completed. You can specify commands directly or execute a .BAT file script, PERL script, or other command processing program. By default, the Cycle End command is launched immediately. Or you can specify that the Cycle End command be launched at a random time within a defined processing window (up to 60 minutes in duration).

Processing multiple files.

If the Collection service was restarted (due to a re-boot, for example) during the last collection cycle, there will be multiple .smf collection files in the\data\Previous folder at the time the Cycle End Command you specified is launched. Consequently, the Cycle End processing you perform should allow for the fact that multiple collection files may be available for processing. A wildcard in your filespec normally accomplishes this.

For ease of processing in MXG, you will normally consolidate collection files representing many different machines into one file. Use the COPY command available in Windows NT/2000 with the /B option to consolidate multiple Performance SeNTry collection data files into one large file for processing:

COPY /B C:\NTSMF24\data\Previous\*.SMF C:\NTSMF\Daily\NTDaily.SMF

The /B option of the COPY command concatenates multiple files without terminating each file with a x’3F’ end of file character. In the above example the file named NTDaily.SMF is associated with the MXG NTSMF input file name, for example. To process this data file in MXG, you would code a SAS language FILENAME NTSMF specification that references C:\NTSMF\Daily\NTDaily.SMF if you are running SAS under Windows NT/2000.

If you are copying files to send to an MVS mainframe for processing using SAS, use a combination of /A and /B switches, as follows to prevent embedded EOFs in the concatenated file:

copy /a f1.smf+f2.smf+f3.smf bigfile /b

Using the /a will cause the EOF mark to be removed from each of the source files f1.smf through f3.smf as they are concatenated into the destination file bigfile. the /b following the destination file bigfile ensures that an EOF mark is not appended to bigfile and subsequently transmitted to MVS.

More generally, to create a concatenated file usable under MVS:

copy /a *.smf bigfile /b

Using ftp

If you need to send daily .smf files to a non-Windows host machine for processing or the machine you need to transfer the files to is separated by a firewall, you may need to use the ftp utility to copy the files. Here is an example of a simple .bat command file that utilizes ftp:

COPY /B C:\NTSMF24\Data\Previous\*.SMF C:\NTSMF24\Data\Consolidated\Consolidated.SMF

ftp -v -i -s:Transfer.ftp

exit

The following example script connects to a remote server, uses a valid Username and Password to log into the remote system. Once logged unto the remote system, the program changes directory (CD) to the remote directory C:\Consolidated and sends the file from the originating folderC:\NTSMF24\Data\Consolidated\Consolidated.smf to the remote file Consolidated.smf on the remote server. Both origination and remote servers have to have FTP enabled, with common Usernames and Passwords defined. In the script, replace the keywords Remote, Username, and Password with parameters that are valid for your environment.

open Remote

Username

Password

cd C:\Consolidated

send C:\NTSMF24\Data\Consolidated\Consolidated.SMF Consolidated.SMF

Bye

You can also using ftp wild cards to transfer multiple files in a single command execution stream. The easiest way to do this to create an ftp script that uses the mput subcommand to send multiple .smf files contained in the \Previous folder.

CD C:\NTSMF24\DATA\PREVIOUS\

ftp -v -i -s:C:\FTPROCESS\transfersmf.txt

exit

where transfersmf.txt is a text file containing the following ftp script:

Rem Transfersmf.ftp sends multiple files to a remote central server

open Remote

Username

Password

prompt

mput *

Bye

The prompt subcommand turns off ftp interactive mode. The * wildcard filespec means that all data files located in the \Previous folder will be transferred.

Invoking the Summarization utility

The Summarization utility is designed to run at Cycle End, although it can also execute in a standalone environment. Letting the Summarization utility perform aggregation of the detailed collection files, you can distribute a good deal of the load involved in post-processing .smf files across your network.

In this example, the Summary utility is invoked in the Cycle End command script after a step that consolidates multiple detail files, prior to invokingftp to transfer the .sum.smf summarized file to a consolidation folder on a remote server:

copy /b C:\NTSMFv24\Data\Previous\*.SMF C:\NTSMFv24\Data\Previous\Allprevious.smf

rem Next run summarization rtn to summarize in 60 minute intervals

cd ..

cd ..

dmsumsmf.exe -fAllprevious.smf -s60

rem Next execute FTP transfer bat file to upload file

ftp -v -i -s:Transfer.ftp

exit

where Transfer.ftp consists of the following script:

open 10.0.0.7
ftpguest
password
cd transfersmf
send
C:\NTSMFv24\Data\previous\Allprevious.sum.SMF
Allprevious.sum.SMF
Bye

Historical files

You can also take advantage of a roll-up feature in the Summarization utility to create and maintain historical data files that can streamline your reporting procedures. To use the roll-up feature, concatenate the current data file with the previous historical data file. When you invoke the Summary Utility on the combined file, specify the number of consecutive days worth of data that you want to keep. Any data older than the number of days you specify will be dropped from the summarized output file. Using the Summarization utility, you can create and maintain as many historical files as you want, as the example below illustrates. The sample script below maintains a daily historical roll-up file that spans the last 14 days, a weekly roll-up that spans 26 weeks, and a monthly roll-up that spans the last two years.

At Cycle End, issue the following command:

summary-example1.bat @LastDataFile:p @StartupDir

where summary-example1.bat is coded as follows:

rem Summay utility example

rem Cycle End command: summary-example1.bat @LastDataFile:p @StartupDir

rem Maintains daily, weekly, and monthly history files

rem Daily roll-up file: keeps 14 days of history, summarized to 10 minute intervals

rem Weekly roll-up file: keeps 26 weeks of history, summarized to 1 hour intervals

rem Monthly roll-up file: keeps 24 months of history, summarized to 4 hour intervals

rem ******************************************************************

rem

rem Utilizes the following Directory structure:

rem <ntsmf root>\data\

rem – \Archive

rem – \Backup  — backup History files first

rem – \Current

rem – \History — current History roll-up files

rem – \Previous

rem – \Work    — for temporary files

rem

rem summarize the contents of the \data\previous folder — yesterday’s smf data files

copy /B %1\*.smf %2\data\Work\%COMPUTERNAME%.combined.smf

%2\dmsumsmf.exe -f%2\data\Work\%COMPUTERNAME%.combined.smf -k2 -H”%2″ -s10

rem Delete old backups

del del %2\data\Backup\*.old.sum.smf

rem Backup History files

ren %2\data\Backup\%COMPUTERNAME%.daily.sum.smf %2\data\Backup\%COMPUTERNAME%.daily.old.sum.smf

ren %2\data\Backup\%COMPUTERNAME%.weekly.sum.smf %2\data\Backup\%COMPUTERNAME%.weekly.old.sum.smf

ren %2\data\Backup\%COMPUTERNAME%.monthly.sum.smf %2\data\Backup\%COMPUTERNAME%.monthly.old.sum.smf

copy %2\data\History\*.smf %2\data\Backup\

rem

rem Keep 14 days worth of Daily data

rem

rem Step 1: move the Daily summary file to \Work

move %2\data\History\%COMPUTERNAME%.daily.history.sum.smf %2\data\Work\

rem Step 2: Concatenate yesterday’s data with the current Daily history roll-up

copy /B %2\data\Work\%COMPUTERNAME%.combined.sum.smf+%2\data\Work\%COMPUTERNAME%.daily.sum.smf %2\data\History\%COMPUTERNAME%.daily.smf

rem Step 3: Run the Summarization utility on the combined file

%2\dmsumsmf.exe -f%2\data\History\%COMPUTERNAME%.daily.smf -k14 -H%2 -s10

rem

rem Keep 26 weeks worth of Weekly data

rem

move %2\data\History\%COMPUTERNAME%.weekly.sum.smf %2\data\Work\

copy /B %2\data\Work\%COMPUTERNAME%.combined.sum.smf+%2\data\Work\%COMPUTERNAME%.weekly.sum.smf %2\data\History\%COMPUTERNAME%.weekly.smf

%2\dmsumsmf.exe -f%2\data\History\%COMPUTERNAME%.weekly.smf -k182 -H%2 -s60

rem

rem Keep 24 months worth of Monthly data

rem

move %2\data\History\%COMPUTERNAME%.monthly.sum.smf %2\data\Work\

copy /B %2\data\Work\%COMPUTERNAME%.combined.sum.smf+%2\data\Work\%COMPUTERNAME%.monthly.sum.smf %2\data\History\%COMPUTERNAME%.monthly.smf

%2\dmsumsmf.exe -f%2\data\History\%COMPUTERNAME%.monthly.smf -k730 -H%2 -s240

rem

rem Clean up Work & History folders

del %2\data\Work\*.smf

del %2\data\History\%COMPUTERNAME%.daily.smf

del %2\data\History\%COMPUTERNAME%.weekly.smf

del %2\data\History\%COMPUTERNAME%.monthly.smf

Using WSH scripts

Windows Script Host (WSH) scripts written in either VBscript or Javascript provide a flexible alternative to shell scripts. You can specify that a WSH script be launched at Cycle End. Launching WSH scripts at Cycle End requires the 2.4.7.3 version of the collection agent.

The Script Center on the Microsoft TechNet web site is an excellent resource for anyone that wants to learn about WSH script basics. If you have access to the Microsoft Developer Network (MSDN) web, you can access many additional resources for developing WSH scripts.

WSH technology is built into Windows 2000 and above, so there is no extra software you need to start developing WSH scripts. Utilizing built-in scripting objects like Wscript, WshShell, WshEnvironment, and WshArguments, your script can execute other programs, access environment variables and retrieve script arguments. A file system scripting object allows your script to manipulate folders and files. Finally, using WMI, your script can access any of the configuration information that is accessible in the vast WMI repository.

WARNING: When you specify the execution of a WSH script at Cycle End, make sure you specify the //B runtime parameter when you invoke wscript.exe or cscript.exe. The //B runtime parameter runs your script in batch mode, suppressing all screen output and keyboard interaction. If you do not specify the //B runtime parameter and your WSH script issues a Wscript.Echo statement that writes to the display or initiates any other interaction with the screen, your script will hang.

For example, specify the following Cycle End command in your DCS parameters:

Wscript.exe //B “@StartupDir\CycleEndscript1.vbs” /datapath:@LastDataFile:p /rootdir:@StartupDir

For tips on configuring the collection service in order to debug a Cycle End WSH script, see the section entitled Debugging WSH scripts below.

The following VBscript example illustrates the functions typically performed in a Cycle End script using WSH. It performs the following processing steps:

1. It performs some string manipulation of today’s date to create a unique file name.

2. It validates the arguments /datapath and /rootdir arguments that are passed to it from the command line invocation.

3. It calls the NtDACmd utility to update the NTSMF PDB Sql Server database.

4. It then uses the Copy command with the /B option to consolidate multiple files into a single daily file, embedding the current date into the consolidated file’s name. (The file system scripting object has no comparable function.)

5. It calls the DmSumsmf utility to create a summarized version of the consolidated daily data file.

6. Finally, it copies the consolidated daily data file somewhere — Step 6 is the section of the Cycle End script that requires customization for your environment.

VBscript comment statements — any statement that starts with a ‘ character –  are shown in an alternate color in this listing.

‘ VBScript source code
‘ Creates consolidated daily .smf file in /Previous folder
‘ Creates summarized daily .smf file from consolidated file
‘ Calls NtDACmd to initiate an NTSMF PDB database update
‘ Finally, copies the summarized file to a designated location
‘
‘ The script accepts two parameters:
‘ /datapath — the fully qualified path to the \data\Previous\ folder
‘ where yesterday’s .smf collection files are located
‘ /rootdir — the fully qualified path to the NTSMF root folder
‘ where the DmSumsmf and NtDACmd utilities are located
‘ Sample invokation at CycleEnd:
‘
‘ Wscript.exe //B “@StartupDir\CycleEndscript1.vbs” /datapath:@LastDataFile:p
‘ /rootdir:@StartupDir

‘Step 1 — Create a unique file name

Dim CreatedTodayDate
Dim CreatedTodayDateAsString
Dim EditDate ‘mmddyy format from standard Date function in VBscript
CreateTodayDate = Date
CreatedTodayDateAsString = CStr(CreateTodayDate)
‘Wscript.Echo “Today’s Date String is ” & CreatedTodayDateAsString

dim datestrlen
datestrlen = Len(CreatedTodayDateAsString)

dim loc
loc = InStr(CreatedTodayDateAsString,”/”)
EditDate = Left(CreatedTodayDateAsString,loc-1)
if len(EditDate) = 1 then
EditDate = “0″ & EditDate
End if
CreatedTodayDateAsString = Right(CreatedTodayDateAsString,datestrlen-loc)
datestrlen = datestrlen – loc
loc = InStr(CreatedTodayDateAsString,”/”)
EditDate = EditDate & “-” & Left(CreatedTodayDateAsString,loc-1)
CreatedTodayDateAsString = Right(CreatedTodayDateAsString,datestrlen-loc)
datestrlen = datestrlen – loc
EditDate = EDitDate & “-” & CreatedTodayDateAsString
‘Wscript.Echo “EditDate is ” & EditDate

‘Step 2 — Validate the arguments passed to the script
Set WshShell = CreateObject(”Wscript.Shell”)
Set objEnv = WshShell.Environment(”Process”)
LogonServer = objENV(”COMPUTERNAME”)

Dim namedArgs
Dim PrevFolder, RootFolder
Dim objFSO1,objPreviousFolder,fc1,allthefilenames

Set objFSO1 = CreateObject(”Scripting.FileSystemObject”)
Set namedArgs = WScript.Arguments.Named
‘WScript.Echo “datapath specified: ” & namedArgs.Item(”datapath”)
‘WScript.Echo “rootdir specified: ” & namedArgs.Item(”rootdir”)

PrevFolder = namedArgs.Item(”datapath”)
RootFolder = namedArgs.Item(”rootdir”)

‘Change the next assignment statement to generate a different name for consolidated data file
DailyFileName = PrevFolder & “\” & LogonServer & “.” & _
“consolidated_daily_filefor” & “.” & EditDate & “.smf”

‘WScript.Echo DailyFileName

If objFSO1.FolderExists(PrevFolder) then
Else
‘WScript.Echo “Invalid datapath argument: ” & PrevFolder
Return
End If

If objFSO1.FolderExists(RootFolder) then
Else
‘WScript.Echo “Invalid root directory argument: ” & RootFolder
Return
End If

‘Step 3 — Call the NtDACmd to update the NTSMF PDB database

command = RootFolder & “/” & “ntdacmd.exe -d ” & PrevFolder & ” -threads 2″
Dim execCommand2
Set execCommand2 = WshShell.Exec(command)

‘Wscript.Echo execCommand2.StdOut.ReadAll

‘Step 4 — Execute the Copy coammnd to consolidate the daily .smf data files

dim copyspec
copyspec = “”"” & PrevFolder & “\*.smf” & “”"”
‘Wscript.Echo “Copy .smf Files in Previous Folder to: ” & DailyFileName

dim command
command = “cmd.exe /c ” & “”"” & “copy /b ” & copyspec & ” ” & DailyFileName & “”"”
‘Wscript.Echo “Copy command string: ” & command

Dim execCommand
Set execCommand = WshShell.Exec(command)

‘Wscript.Echo execCommand.StdOut.ReadAll

‘Step 5 — Execute the DmSumsmf utility to summarize the daily .smf data files

command = RootFolder & “/” & “dmsumsmf -f” & DailyFileName & ” -h” & RootFolder
‘Wscript.Echo “Dmsumsmf command string: ” & command
Dim execCommand1
Set execCommand1 = WshShell.Exec(command)

‘Wscript.Echo execCommand.StdOut.ReadAll

‘Step 6 — Finally, copy the consolidated summary file somewhere ===>
‘Careful. Multiple sum.smf output files may be created
command = “cmd.exe /c ” & “”"” & “copy ” & PrevFolder & “\*.sum.smf \\STPUTL04\NTSMF” & “”"”
‘Wscript.Echo “Copy command string: ” & command
‘Dim execCommand3
‘Set execCommand3 = WshShell.Exec(command)
‘Wscript.Echo execCommand3.StdOut.ReadAll

Debugging WSH scripts. An old fashioned, but effective, way to debug WSH scripts is to use the Wscript.Echo method to write to the screen to trace the flow of program execution. This sample script is laced with Wscript.Echo  statements that were used in debugging, but subsequently have been commented out. In order to view the output from Wscript.Echo statements while you test your script during Cycle End processing, you must

	remove the //B parameter that suppresses interactive output from the Wscript invocation statement on the Cycle End command line,
	allow the Performance SeNTry collection service to interact with the desktop (as illustrated below), and

restart the Performance SeNTry collection service after you have applied this Service Control Manager parameter change so that it can take effect.

This will allow you to view all Wscript.Echo output when the script executes during Cycle End command processing.

When you execute Wcsript.exe or ccript.exe to run your script, the  //B parameter indicates batch execution mode. Batch mode suppresses command-line display of user prompts and script errors, which is important when you run a script during Cycle End command processing. Without the the //Bparameter, your script will hang, waiting for user input if you execute a Wscript.Echo statement that writes to the screen.

WARNING: When you have finished debugging your script, make sure you restore the //B parameter to the script invocation command line before you distribute to a production server. If you do not allow the collection service to interact with the desktop, as illustrated above, any attempts to write to the screen using Wscript.Echo statements will be invisible. In this case, not only will the script hang, but you will not be able to tell that it is hung.

]]> http://faq.demandtech.com/2010/03/04/will-you-please-explain-cycle-end-processing/feed/ 0 http://faq.demandtech.com/2010/02/22/counter-values-you-need-are-not-available-in-the-ntsmf-data-collection-file/ http://faq.demandtech.com/2010/02/22/counter-values-you-need-are-not-available-in-the-ntsmf-data-collection-file/#comments Mon, 22 Feb 2010 18:30:41 +0000 admin http://faq.demandtech.com/?p=185 This procedure is for diagnosing data collection problems where the Counter values you need are not available in the NTSMF data collection file and not visible in System Monitor either. If you are still not able to collect the missing Objects and Counters after you have followed all the steps in this procedure, please contact Customer Support and tell us about your problem. Please attach the .smf data file and the <computername>.ntsmf.log file from the machine where you are experiencing the problem.

1. Is the Counter specified for collection, but not available for collection on the machine?

An Event ID 604 Warning message in the Application Event log (or the <computername>.ntsmf.log file) documents performance Counters that were specified in the DCS Data Definition, but are not available for collection on the local machine. This happens because not all the Counters associated with a given performance Object are available for collection across all versions of the associated Performance Library DLL that is responsible for maintaining them. While the developers of the operating system base Counters, like Processor and System, are pretty good about maintaining upward compatibility between versions of their Perflib DLLs, not all developers of Perflib DLLs are as circumspect.

For instance, if you specify that you want to collect all the Counters in the Internet Information Services Global Object and you are running IIS version 5.0, you are likely to receive a Warning message similar to the following:

10/30/02-09:06:00 – Event ID: 604, Category: DCS, Severity: Warning

The following global counters were specified in the DCS but not found on the local machine and will be skipped.

Object, Internet Information Services Global (L2596, G4022)

Counter, Cache Size (L0, G4024)

Counter, Cache Used (L0, G4026)

Counter, Cached File Handles (L0, G4028)

Counter, Directory Listings (L0, G4030)

Counter, Objects (L0, G4032)

Counter, Cache Flushes (L0, G4034)

Counter, Cache Hits (L0, G4036)

Counter, Cache Misses (L0, G4038)

Counter, Cache Hits % (L0, G4040)

Counter, Measured Async I/O Bandwidth usage/ (L0, G4052)

These are Counters that used to be in the Internet Information Services Global Object, but were dropped in later versions of IIS.

This situation is not an error condition (which is why the Collection service only warns you about it), but you need to abandon any hope you may have of collecting those specific Counters on this machine.

1. Is the Performance Library DLL associated with the missing data missing?

Sometimes the Performance Library DLL that is defined is simply missing on the local machine. When the Performance Sentry collection service attempts to load the Library module referenced in the HKLM\SYSTEM\CurrentControlSet\Services\<servicename>\Performance key, the module is not found. In this event, you should see an Application Event log similar to the following:

11/05/02-17:48:08 – Event ID: 210, Category: Discovery, Severity: Warning
The Performance Library, “Perfctrs.dll”, for service “TCP/IP Protocol Driver” (Tcpip), failed to load.
The system error was “The specified module could not be found (126)”.
Performance data for this service will not be available.
The Event Viewer (source Perflib) may have more details

In this example, the perfctrs.dll module identified in the Library field of the /Services/Tcpip/Performance subkey was missing. When the collection service tried to load it, the Load operation failed, for the following reason: “The specified module could not be found.” When a problem like this occurs, using either the Extended Counter List applet or the Registry Editor, determine where the Library is supposed to be and verify that the Perflib DLL is, in fact, missing. If the Library field of the /Performance subkey points to a fully qualified filename, look for the module in the folder specified. If only the module name is referenced, then look for it in the %system32% folder in the Windows <root> directory.

Sometimes this occurs when the application that installed the Performance Library was removed, but the corresponding Performance subkey was not deleted when the uninstall procedure executed. In this case, it is usually safe to remove the Performance subkey manually using the Registry Editor. The Event ID 210 Warning message will disappear after that.

However, if you feel that the missing Performance Library DLL is a mistake, try to correct the problem. Sometimes uninstalling and re-installing the application will restore the Perflib. Alternatively, try to locate a working version of the Performance Library DLL on a different machine running the same or a similar level of software. Copy that Perflib DLL module to this machine and then restart the collection service to see if that solves the problem.

1. Is the Performance Library DLL associated with the missing data defective?

When the collection service encounters a defective Perflib DLL, it generates Application Event log messages that help to document the data collection problem. Common problems that you might see reported include:

1. a Perflib DLL that does not initialize successfully,

2. a Perflib DLL that fails to return any Objects and Counters, and

3. a Perflib DLL that returns zero values for all the Counter values that it reports.

Sometimes, the Perflib DLL has a bug which causes an unhandled Exception or other error condition that causes the collection thread calling the Module to terminate. Prior to terminating, the collection service (assuming you are running the “fat” version of the collector) will generate a Stack trace showing the precise status of the collection thread at the time the error occurred. This diagnostic trace information should be forwarded to the party responsible for the Perflib DLL that failed for resolution.

An example diagnostic trace message generated by an unhandled exception in a Perflib DLL is illustrated below:

As a result of encountering a serious error condition related to calling a defective Performance Library DLL, the collection service will recover and continue to gather data from whatever Perflib DLLs that are functioning correctly for the duration of the collection cycle. While the collection service bypasses collecting data from the defective Perflib for the remainder of the collection cycle, it will automatically retry the module in the initialization phase at the beginning of the next collection cycle. Consequently, the Warning messages that document the problem are likely to recur regularly at the beginning of each collection cycle until you decide to investigate and resolve the problem.

Only the most serious Perflib DLL defects trigger a diagnostic trace message. You are much more likely to encounter less serious defects like the ones described below.

Sometimes the collection service is able to locate and load the Perflib DLL without problems, but cannot gather the performance data the Perflib was designed to collect. Consider that many Perflib DLLs associated with extended Counters are designed to gather data from inside servers application processes like sqlserver.exe, store.exe, and inetinfo.exe. A frequent cause of data collection problems is that the process that a Perflib DLL needs to acquire performance data from is not running. For example, if the sqlserver.exe process is shutdown, the sqlctr80.dll Perflib DLL designed to gather SQL Server statistics will report an error condition, which the collection thread will post to the Event log.

If sqlserver.exe stops running or is hung for some reason, failed attempts to gather data SQL performance data from the SQL Performance Library DLL (e.g., sqlctr80.dll) will generate Warning messages like the following for each SQL performance Object the collector has been instructed to collect:

11/05/02-22:31:00 – Event ID: 2200, Category: Collection, Severity: Warning
Object, SQLServer:General Statistics (L2346, G17300), failed to return data this interval

11/05/02-22:31:00 – Event ID: 2200, Category: Collection, Severity: Warning
Object, SQLServer:Databases (L2370, G17700), failed to return data this interval

11/05/02-22:31:00 – Event ID: 2200, Category: Collection, Severity: Warning
Object, SQLServer:SQL Statistics (L2470, G18300), failed to return data this interval

Errors can also arise when the process that the Perflib DLL must communicate with to gather performance data is simply unavailable. In this example, the w3ctrs.dll Performance Library DLL cannot access the Inetinfo Web server publishing service that is the source for the Web service performance data that it reports.

11/05/02-16:50:16 – Event ID: 1309, Category: PerfData, Severity: Warning
World Wide Web Publishing (Global): no objects were found in the performance data buffer

11/05/02-16:51:16 – Event ID: 96, Category: N/A, Severity: Warning
The following performance libraries failed to return data.
Data from objects from these libraries will be unavailable for this cycle.
World Wide Web Publishing (W3SVC), “w3ctrs.dll”

11/05/02-16:51:16 – Event ID: 2200, Category: Collection, Severity: Warning
The following objects were specified in the DCS but failed to return data.
These objects will be skipped this cycle.
Web Service (L0, G8300)

11/05/02-16:51:16 – Event ID: 2200, Category: Collection, Severity: Warning
Object, Web Service Web Service (L0, G8300), failed to return data this interval

To rectify this situation, you need to resolve the underlying problem that is preventing the process the Perflib DLL depends on from running.

Version 2.4.7 of the Performance SeNTry (NTSMF) data collector will not execute if DEP is active. This does not apply to versions after 2.4.7

For details and options, see this Technical Note.

The main dispatcher thread of the Collection service generates a separate collection thread for each Perflib DLL that it finds installed on the local machine during the Discovery phase at the beginning of every collection cycle. (Follow this link for more .) Each Open and Collect call to a Perflib DLL is performed under a dedicated collection thread. With the threaded collector, it is possible to detect and recover from most failures involving direct calls to Perflib DLLs, sometimes even when calls via the Microsoft PDH and WMI plumbing that other products use, including System Monitor, fail. Under normal circumstances, the NTSMF collection service continues to gather performance data even in the face of buggy, third party Perflib DLL modules.

There are extenuating circumstances when it is desirable for the NTSMF collection agent to ignore completely specific Performance Library DLLs that are defective and perform no processing on these modules. Performance Library DLLs that malfunction in the following ways are candidates for exclusion from all NTSMF processing :

• The Perflib fails to return properly from any Open or Collect call and causes the collection agent, the application being monitored, or the system to hang.
• When NTSMF successfully opens the Performance Library DLL, the module cannot be shared properly with other applications that also attempt to Open them.

When defective Perflibs are excluded from processing, the NTSMF collection agent does not attempt to load the DLLs or call their Open routines during the Discovery phase at the beginning of every collection cycle. You exclude these defective Perflibs from all NTSMF-related processing using entries in the DmPerfss.cfg configuration file.

Note: the use of the the DmPerfss.cfg configuration file is optional. A default DmPerfss.cfg configuration file is installed automatically when you run the standard software Setup routine. The default DmPerfss.cfg configuration file contains entries for Performance Library DLLs that we have seen cause severe problems in some of our customer sites. We recommend that you install and use the default DmPerfss.cfg configuration file.  You can modify the defaultDmPerfss.cfg configuration file using any text editor if you determine there are additional Performance Library DLLs that need to be excluded from NTSMF processing. We recommend that you contact Customer Support before you make modifications to the DmPerfss.cfg configuration file for your environment. If you are having a problem, there may well be other solutions to consider. If the Performance Library DLL is defective, we might like to let other customers know about it.

The <Perflibs> section of the DmPerfss.cfg configuration file can contain an Exclude list and an Include section. These sections identify specific Performance Library DLLs by module name (and optionally by version) that are to be excluded or included for processing at the beginning of a collection cycle.

Exclude list. The Exclude list causes the collector to exclude the modules listed (by version, if specified) from all NTSMF collection agent processing. The collection agent will not attempt to load a Perflib DLL module on the Exclude list at the beginning of a collection cycle. The Version specification is optional. Module entries without a version specification cause all versions of the Perflib DLL to be excluded. If specific Perflib DLL module versions are to be excluded, then they must be listed in the Version section. If there are duplicate entries, the Exclude list takes precedence over modules on the Include list .

Code a <Module FileName=”filename“></Module> section for each Performance Library DLL that you want to exclude from processing. Optionally, you can add one or more <Version> specifications that qualify which versions of the Performance Library DLL should be excluded. If the <Version> specification does not exist, any version of the Performance Library DLL that is found is excluded.

An example DmPerfss.cfg configuration file with an Exclude list is coded below:

<PerfLibs>

<Exclude>
<! — Sybase ASA object –>
<Module FileName=”dbctrs8.dll”></Module>

<!– .NET CLR Data, NET CLR Networking –>
<Module FileName=”netfxperf.dll”><Version>1.0.3215.11</Version></Module>

<!– .NETFramework –>
<Module FileName=”mscoree.dll”><Version>1.0.3215.11</Version></Module>

<!– FileReplicaConn, FileReplicaSet –>

<Module FileName=”NTFRSPRF.dll”><Version>5.1.3590.0</Version></Module>

<!– WmiApRpl –>
<Module FileName=”wmiaprpl.dll”><Version>5.1.3590.0</Version></Module>

<!– Lotus Notes –>
<Module FileName=”nnotes.dll”> </Module>

</Exclude></PerfLibs>

Text inside a <! This is a comment > Comment section is ignored.

During the Discovery phase at the start of every collection cycle, the DmPerfss.cfg configuration file is processed to determine what Performance Library DLLs should be excluded from processing during that cycle. An Event ID 214 Informational message is written to the <computername>.ntsmf.log file that documents the Performance Library DLLs that were specifically excluded from processing, as illustrated below:

08/02/04-00:00:11 – Event ID: 214, Category: Discovery, Severity: Info

The following Performance Libraries are excluded from processing as specified in DmPerfss.cfg. Performance data from these libraries will not be available this cycle.
.NET CLR Data (.NET CLR Data), “netfxperf.dll”, version 1.1.4322.573
.NET CLR Networking (.NET CLR Networking), “netfxperf.dll”, version 1.1.4322.573
.NETFramework (.NETFramework), “mscoree.dll”, version 1.1.4322.573
WmiApRpl (WmiApRpl), “C:\WINDOWS\System32\wbem

Error Messages. In the case of errors accessing the DmPerfss.cfg configuration file, the configuration file is ignored and collection cycle processing proceeds as if theDmPerfss.cfg configuration file did not exist.

If the the DmPerfss.cfg configuration file is invalid, then the following Warning message is written:

08/02/03-00:50:13 – Event ID: 96, Category: None, Severity: Warning
The configuration file failed to open.
The Inclusion/Exclusion list will not be used this cycle.

If the DmPerfss.cfg configuration file contains syntax errors, then the following Warning message is written:

08/02/03-00:50:13 – Event ID: 96, Category: None, Severity: Warning
Failed to load, “E:\DmPerfss 2.4.6\bin\Debug\DmPerfss.cfg”.
Parse error, 0xC00CE503, encountered on line, 25, at position, 14.
Reason = Incorrect syntax was used in a comment.

For customers who do require less than 60 second sample intervals we recommend that they pare their data collection sets down to the absolute minimum number of counters for collection.   These counters usually center around a specific subsystem (like disk) or processes for a specific application.

Some customers have designed these collection sets to run 24×7, however, most customers will set this up for only a short period of time while they are doing a performance study.

Unfortunately, there is not much written that documents AD performance issues and how to use these counters. This TechNet overview article is a good place to start. There are two main objects to monitor:

1. Information related to the NTDS Object:

http://technet2.microsoft.com/windowsserver/en/library/6a2abace-5fab-49f0-8c6d-5daea891a5f71033.mspx?mfr=true

2. Information related to the Database Object:

http://technet2.microsoft.com/windowsserver/en/library/26bfde1e-7938-4573-b12f-6e4e9cedf4a51033.mspx?mfr=true

The following represents Microsoft’s recommendations when monitoring Domain Controllers:

http://technet2.microsoft.com/windowsserver/en/library/c5d72b6f-5974-4263-b29f-2eece0ab44371033.mspx?mfr=true

It will at least help you get acquainted with the basic terminology that AD uses.

04/14/08-13:44:04 – Event ID: 2900, Category: Perf Text, Severity: Error Insufficient data in performance name table

Concerning the message “Insufficient data in performance name table” refers to the file perfc009.dat that resides in the <Windows>\system32 folder. This file contains the names of all installed objects and counters. The name table can become corrupted during installation of new or modified Counter definitions using the lodctr utility.  Even though Microsoft enhanced the procedure to rebuild a  damaged perfc009.dat file in Windows Server 2003 ( KB 300956)  and these problems typically impact all consumers of performance counters, the Performance Sentry Collection Service was modified to recover from severe damage to the perfc009.dat that it finds during Discovery and continue processing.

As an alternative, to the Microsoft KnowledgeBase entry, we sometimes suggest to our customers that they simply copy a Perfc009.dat file from a similarly configured server (after renaming the existing corrupt Perfc009.dat file – for backup or supporting documentation for the error).You can verify the fix by starting Perfmon (Performance Monitor) and looking at the performance counters.

Then stop and restart the Performance Sentry Collection Service so that service will re-initialize using the new, or repaired, file.

For additional repaired options you can read Microsoft’s knowledge base article: KB 300956

Ever notice all those Messages that the Performance Sentry collection service writes to the Application Event log and the <computername>.ntsmf.log file? These messages are designed to help diagnose data collection problems. By reviewing the Perfomrance Sentry Collection service Application Event log Warning messages that document most of the more common data collection anomalies that occur, you can usually identify and resolve a data collection problem quickly. This FAQ tells you how to use these Warning messages to solve data collection problems.

Please follow this simple procedure whenever you have a problem collecting data. First, answer the question below, and proceed to the next page of instructions based on your answer.

Is the Counter you need visible using the Microsoft System Monitor?

Start an interactive System Monitor session and see if the Counter or Counters you need are visible.
If the Counter you need is visible in System Monitor, the Performance Sentry Collection Service should also be able to collect it.

If the Counter value you need is not visible in System Monitor, the Performance Sentry Collection Service probably cannot collect it either. However, it is quite possible that Performance Sentry will generate a diagnostic  error or Warning message that will help you understand why the data you are interested in cannot be collected.

Note that beginning in Performance Sentry version 2.4.5, the Performance Sentry Collection Service does not honor the Disable Performance Counters Registry flag when it is set. Neither will any action performed by the Performance Sentry Collection Service ever cause the Disable Performance Counters Registry flag to be set. However, when the Disable Performance Counters Registry flag is set, it may indicate there is an underlying problem with a Performance Library DLL that could effect Performance Sentry’s ability to gather the associated performance data.

Instead of Disable Performance Counters, Performance Sentry version 2.4.6 and higher utilizes a comparable facility that allows you to specifically exclude Performance Library DLLs that are troublesome. Specific Performance Library DLL excludes and includes are noted in a configuration file in the root NTSMFxx folder named DmPerfss.cfg.

Running Performance Sentry under MS Cluster Server

To run Performance Sentry under a Windows cluster environment:

1. Install the Performance Sentry Collection service in each member of the cluster. There is no need to define any Cluster Resource as a “Generic Application” or “Generic Service”. Performance Sentry will automatically begin to read all shared disks and resources defined by the owner of the cluster on the next collection interval when a failover occurs. MS Cluster Server is started as a service and it may take longer to start than other services in a cluster.

It is recommended that the Performance Sentry service starts after the MS Cluster Server is started so that Performance Sentry can “see” the presence of cluster share drives and logical disk in the respective nodes. This service dependency can be accomplished by adding a “DependOnService” registry value for the particular cluster server software under:
HKLM\System\CurrentControlSet\Services\Dmperfss