“We were very surprised by the ease of setup and the depth of monitoring possible with NetCrunch. NetCrunch finds a problem and tells us what that problem is before we even see an interruption in the data-stream.”
Underwater research equipment
Brian Chee from University of Hawaii talks about the making of this extraordinary project. “The typical reaction of the science community for monitoring has been some sort of 'stovepipe' (aka custom) application, but in the case of ACO, some of the goals were to dramatically reduce both the cost of setting up the observatory, but also reducing operational costs. In this case the answer was to reduce customized components to an absolute minimum and whenever possible to use commercial off the shelf (COTS) components and software. Having the ability to leverage the COTS diagnostic tools through SNMP, ICMP and WMI is increasingly allowing us to predict potential problems and prevent failures.
IT personnel working on the project were very surprised by the ease of setup and the depth of monitoring possible with NetCrunch. Previous experience with products like HP OpenView colored their experience and was why aggressive SNMP based networking monitoring was originally not proposed, both because of cost, but also massive training costs. In this case the NetCrunch system was originally setup by one of the Oceanographers, rather than the IT personnel.”
“The default (network monitorng solution) has been What’s up Gold for many years, however some of the science investigators have tired of the price increases without much in the way of capability increase. The team wanted a system that had a web portal capability that would also fit into our very tight budget” - he continues.
"We have custom scripts that monitor data transfer between our seafloor instruments and our shore-based data acquisition servers that send email alerts when new data have not arrived within a certain period of time. With the new updates and powerful tools included in NetCrunch 9, it is common for NetCrunch to notify me of a network interruption before our in-house scripts do. In other words, NetCrunch finds a problem and tells me what that problem is before we even see an interruption in the data-stream. Daily, weekly, and monthly device traffic reports allow us to analyze our network traffic patterns to improve network performance, and nightly disk usage reports help us monitor our limited data storage system resources. NetCrunch has significantly improved our ability to monitor our undersea data network, troubleshoot network errors, and allows us to continually improve the flow of data from the seafloor to scientists around the globe." - adds Kellen Rosburg, Oceanographic Data Specialist.
NetCrunch capabilities have grown substantially, and the user interface has improved remarkably. Kellen Rosburg, who joined Aloha Calbed Observatory a bit over a year ago, was able to develop new reports for daily, weekly and monthly monitoring that were essential in identifying network issues and changes to device configurations that notably improved performance. In one particular case, he was able to identify the impacts of a failed (flooded instrument module) serial interface that was randomly generating signals that a device on the ACO interpreted as data. NetCrunch turned out to be vital is analyzing in real-time experimental configurations to test the hypothesis regarding this signal source which "couldn't" have been operating.
“The ability to monitor combinations of both ports and services are slowly being deployed deeper and deeper along the undersea cable. Our combination of dry land equipment and undersea equipment is reflected in NetCrunch Tabs and grouping, giving us the ability to very quickly see where in the component chain problems are occurring. Additionally, the ability for the Grouping of components by power boundaries also helps us separate undersea power issues from networking issues on individual components.” - Mr. Chee summarises.
The Aloha Cabled Observatory is a research project of the University of Hawaii School of Ocean and Earth Science and Technology (SOEST) using a re-tasked undersea telecommunications cable (HAW-4) for undersea observational science. The array of hydrophones, cameras, and sensors for measuring pressure, temperature, salinity and currents are just the starting point, since this undersea observatory has the ability to host additional experimental science platforms providing both power and communications. Because of this multi-tenancy operation, the science team was driven to aggressively monitor the entire ACO network with the goal of reducing the possibility of losing invaluable observational data, not to mention that service calls are extremely expensive due to the long lead times necessary to schedule a Remotely Operated Vehicle capable of operating 3 miles underwater. There are 55 undersea IP addressable devices, and 50 on shore.
