Contributed by Todd Williams, ACORN-NS
Across the globe, National Research and Education Networks (NRENs) rely on a shared monitoring framework, called perfSONAR, to monitor their network performance. This framework provides “federated” coverage of paths across different networks and geographic borders. That means it gives better end-to-end updates to individual networks of their performance and helps to isolate problems that may be affecting everyone.
The Canadian NREN has been using perfSONAR for several years. CANARIE has supported the purchase of standardized equipment and implementation by each NREN partner, to ensure a common focus on maintaining the performance of this fast, reliable network. In this post, I will break down the experience by ACORN-NS (the research and education network provider for Nova Scotia) of using perfSONAR to test the network speed for Cape Breton University. The results of this project could be useful to other institutions who are looking to maximize their research network capabilities.
Background
As part of our NREN perfSONAR speed testing project, ACORN-NS acquired two servers. One was sent to the gigapop (gigabit point-of-presence) at the Dalhousie Studley Campus in Halifax, to remain in stationary use there. The plan for the second server was for it to be moved around to different member and affiliate locations, to run remote testing as required.
In December 2018, the mobile testing server was shipped to Cape Breton University (CBU) in Sydney, where the CBU technicians, in collaboration with the ACORN-NS team, successfully ran the prescribed tests.
However, we realized that more testing (on a more permanent basis) would provide much greater benefit to both the university and ACORN-NS. We have now decided to purchase additional servers to deploy at our 11 member institutions, so that each can continue to operate its own monitoring system. This will have the added bonus of allowing ACORN-NS to set up a permanent dashboard and provide regular reporting on the performance of the network within Nova Scotia, while still contributing to the national dashboard with CANARIE and other NREN partners.
The Results from Cape Breton University
The purpose of having a perfSONAR instance at CBU was to determine the real-world health and speed of the ACORN-NS network. This, in turn, would help identify deficiencies and provide guidance for future upgrades.
Testing Methodology
A 1U Dell Server was shipped to CBU and connected via a 1GbE connection within a single hop of the
ACORN-NS network, and outside any traffic shaping or firewall devices. Bandwidth and latency tests between CBU and Halifax were scheduled every 4 and 6 hours, respectively, and testing ran over the period of a week.
Test Results
As you can see from the scales in the two graphs below:
- Overall, there was little evidence of any deficiencies at 1GbE between CBU and Halifax.
- Throughput was between 83-92% theoretical line speed, which is near the maximum expected for a majority of these tests.
- The graph shows a few “dips,” which likely represent increased traffic / load either at the Halifax or the CBU end, but these are the exception.
- Latency between the two sites was also near the maximum expected, based on the number of hops and the speed of light.
- The average latency was 4.5ms, with a spike of over 5ms, and one instance where packet loss was up to 1%.
The Need For Further Testing- Next Steps
The connection between CBU and Halifax, when measured at 1GbE, showed no evidence of issues and performed as expected, with minor variations that one would normally see on a live network. However, we quickly realized that this data has limited value, as it only reflects a 1GbE connection. Our network should theoretically be able to achieve a higher speed.
We are now planning to run further tests of the full link speed of the ACORN-NS link, with each member institution, to ensure we are getting a proper picture of our network’s real-world performance and capacity. This activity, including procurement and implementation of the small-form-factor devices, is scheduled for summer and fall 2019.
Lessons Learned
Running this testing was a relatively easy exercise. Creating a virtual machine (VM) environment was easier than shipping the hardware remotely for the one-time testing activity. This is what led us to look at a more permanent solution for each site.
To perform the tests, the sender and receiver had to maintain the minimum connection speed on each end. For ACORN-NS institutions testing to their core, this was 1-10 Gb. To test the connection nationally, none of the members (except Dalhousie/ACORN-NS) have the ability to scale on their own beyond 10Gb, to achieve the 20+ Gb upper limit of the CANARIE connection.
Advice for other institutions running perfSONAR tests?
Institutions looking to utilize perfSONAR should take advantage of national standards for hardware and software, and the testing requirements currently being developed by the NREN Tech Committee. There is currently an ansible standard hardware installation routine being developed that should be useful once available.
Finally, to facilitate a more comprehensive test, it is advisable to increase server throughput to match the national NREN minimum testing requirements, which prescribe scaling beyond 10Gb.