Data Type

Integer

Default Value

0 for FMSlib,
1+2+32+64=99 for MatrixWarrior

Description

This is the master flag for activating the Dashboard performance reports, which generates WEB HTML pages showing the details of your machine and application as it runs. The options set bit flags. Select the value of IWATCH by adding the individual options you want. The following options are available:

  • 0
    Do not generate a performance report.
  • 1
    Generate and update performance report pages for the current running job.
    New pages overwrite old pages. Where appropriate, pages are set to automatically refresh in your browser based on the amount of time since the last update. Automatic refresh is turned off for all pages when the job terminates.
  • 2
    Include subprograms in the performance chart on the Performance page.
    These include matrix multiply, triangle solve, diagonal block factoring, etc. By default only the total performance is listed.
  • 4
    Archive the last performance page at the end of each subroutine.
    A unique file name is generated that includes the date, time, subroutine being monitored and the HTML page name. This option can be useful if several FMS subroutines are called and you want to examine the performance at the completion of the job. Archived pages have file names in the form 
    YYYYMMDDHHMMPage.html
  • 8
    Archive all pages at the end of each subroutine.
    This is similar to option 4 except all pages are archived.
  • 16
    Generate a movie of the performance pages.
  • 32
    Generate a picture of the matrices being processed.
  • 64
    Include a table showing the time spent in the subprograms.
  • 128
    Include the most recently called function at the top of the table.
  • 256
    List the pages as they are updated in the console window.
  • 512
    Skip updating the Disks page. Gathering information from the operating system on disk usage can take time. If intermediate renderings of the Disks page are not required, setting this flag will skip updates. The Disks page is always displayed at the start and end of the job.
  • 1024
    Skip updating the GPU-Dyn page. Gathering information from nvidia-smi on GPU usage can take time. If intermediate renderings of the GPU-Dyn page are not required, setting this flag will skip updates. The GPU-Dyn page is always displayed at the start and end of the job.
  • 2048
    Execute commands at startup by calling popen to obtain system information. This results in a fork of your process, which may fail if your application has allocated a significant amount of memory. These calls occur before FMS allocated it's memory.
  • 4096
    Include the diagonal blocks in the Component Performance table.

There is a two step process for generating and displaying the performance report pages.

  1. WEB page generation.
    The instrumented machine generates WEB pages in HTML format to a directory you specify with the FMSHTML environment variable. If FMSHTML is not specified, the WEB pages are created in your current working directory. The content of these pages is controlled by this IWATCH Parameter. Separate pages are available which describe the machine, CPUs, GPUs, Memory, Disks, Files, Software, Call history, FMS Parameters, Performance, Usage (Linux), Links and Help. As your application runs, new data becomes available and the pages are updated. By default new pages overwrite old pages. In order to make this process more efficient, pages are updated at an interval, rather than continuously, as data changes. This interval is specified by the FMS Parameter NSUPD, which specifies the update interval in seconds.
  2. WEB page display.
    Using a standard WEB browser you display the WEB pages which are stored in the FMSHTML directory. This may be the same as the instrumented machine or any capable WEB device with access to the FMSHTML directory. This browser machine does not communicate back to the instrumented machine. It simply views what is written to the files. As the files are updated, the browser needs to reload the updated pages. This occurs automatically by the browser polling the files at predetermined intervals. The polling interval is set within the page when it is generated, based on the FMS Parameter NSREF and how frequently the page is being updated. You may also force a reload of the page by entering Control-r.

For a general discussion on the IWATCH performance report, see Chapter 7 of the FMS manual.