Description of DSView and Instrument Status Tracking Tools

From Nanuna internal
Jump to: navigation, search

Data Systems Status View and Instrument Status Tracking

There are two online tools for checking ARM instrument status: the Data Systems Status View (DSView) tool, and the IST (Instrument Status Tracking) tool. DSView has the option of using DMF, or site, information, or both.


IST is actually a subset of DSView that allows long term tracking of instrument status. IST uses data and logs that have arrived at the DMF. Usually David Swank, or one of the other people in the SDS group at SGP, updates the status text in IST. The department email for the SDS group is IST does not automatically refresh its screen. There is no history stored for these status statements. The daily SDS Site Status Report is driven by the IST information. In IST, you can save (bookmark) links to specific views down to the instrument per site level. Here are site links for the NSA and OLI IST tools.


DSView is a flash-based web page that shows the current status of instrument data flows at both the sites and the DMF. It auto-updates and therefore is a snapshot in near-real-time. It has multiple options to configure the display to where it is the most useful to the user. Each site has its own DSView, and there is also a master that shows all the sites.

Looking at just the DSView page for a site such as NSA, there are three buttons in the upper right, Disk Swap, Status, and History. Just above the instrument status display of colored circles, there are three buttons, Dense View, Show All, and Show List. On the left side there is a frame with sections for Instruments, Sites, Processes, Statuses, and two collapsed areas for Status Duration and Data Updating.

First, is the Disk Swap button. Most of the sites have a disk swap computer for transferring large data sets that would otherwise choke the internet link. Hovering over this shows the disk swap status, how many files, free space on the swap disk, etc. Normally the site operatons people focus on collections disk.

The History button may not work on the site DSView pages. It may be only on the main DSView page, but it shows if there are times when the collection process fails. Many daylight instruments will "fail" at night, but collections resume at dawn.

On the left, under Sites, there are checkboxes. Using NSA for an example, there are NSA(C), NSA(E), NSA(S). NSA(C) is the central site. NSA(E) is extended facilities, and NSA(S) is for central site stuff that is separated from the main central facility. Many times this will be something like an AOS package that is a couple of km from the central facility. There is also an NSA(X) site. NSA(X) is for instrument systems that push data to the collector, and are guest systems ARM does not control. OLI has OLI(M), for Mobile.

Under Processes, there are checkboxes for (Local) and (DMF) processes. We normally uncheck all the DMF stuff, as they are Someone Else’s Problem (SEP). There are no DMF processes at the sites. Once the data is collected at the site, a transfer program will move it to the DMF. The DMF processes are other people’s problem because we do not have access to any of their systems.

Finally, under Statuses, normally Disabled, and Auto Disabled can be unchecked unless you are looking to see who has turned something off. A process will be Auto Disabled if it is not supported by the instrument. Usually, this will be time-checking. Comments on disabled stuff are manually entered, so what you see is what you get. In almost all cases, when something is disabled, the mentor and the site techs know about it ahead of time, if they are not the one requesting something be disabled.

Most, but not all, instruments support time checking. It's not always an NTP daemon. The loggers that are collected directly (not through LoggerNet) are time-checked by the collection process, but they don’t run NTP.

Collection is pulling the data off the instrument, ingest is post-collection processing to put the data in a format suitable for use. It is only turned on at a site for specific instruments, under specific circumstances, usually when the mentor needs to see the data immediately, and the link to the site is so slow it is not practical to wait for the data to go to the DMF for processing.

If you pick an instrument, and click on its dot in the main frame, a frame will pop up on the right to show detail.

Once you get the screen exactly how you want it, there is no way to save that configuration. It is possible to click things on and off enough that the screen is useless. Reloading the whole page is the easiest way to get back to a known state.

Daily Monitoring Recommendation

Swank’s recommendation to UAF is to go to the main DSView page, which is hosted at the DMF, turn Dense View off, uncheck all sites but NSA(C), NSA(E), NSA(S), and OLI(M). Under Processes, uncheck all the DMF processes, and the Ingest(Site). Ingest is not normally run at the sites. Under Statuses, uncheck the two for Disabled. This will give you a screen that only shows the items of interest.

Where do DSView and IST get their data?

Each site has a collector and a research system. They share a common network file system. The research system supports the web server for the site DSView page. As the data is collected, the files go to the common area, and the web page mines this to produce the status display. However, the DMF options on DSView, and by extension the IST screen, are looking at collected data and collection logs that have been transferred to the DMF. So, it is very possible, and not unusual, for the status at the DMF to be different from that at the site.

Source: Personal communication with David Swank, IT Manager, ARM SGP

Personal tools