The Valkyrie1564 uses the terminology specified in Y.1564 and the MEF architecture documents it references. A few areas are highlighted below.
“Frames” versus “Packets”
They are the basic units of transmission – and some people call them “frames” and other refer to them as “packets”. But since Y.1564 is pretty consistent in using the “frames” term the Valkyrie1564 will also do so.
“Ingress” and “Egress”
The Valkyrie1564 uses the MEF terminology where directions are seen from the network perspective, i.e. the DUT (Device Under Test), So “ingress” traffic is going in to the network (and thus out of the Xena tester), and likewise for the “egress” traffic.
The various application functions are divided into a number of separate panels. These are contained in the main tabbed panel which takes up the right-center part of the GUI.
The various function panels are described below.
|Defined Services Tree View
||Shows all defined services and UNIs in a hierarchical tree view.
||The default main page shown to the user. This page contains a brief guide to assist you in creating an initial test configuration. You can close this panel once you feel that it is no longer useful to you.
||This panel contains all configuration of folders, services and UNIs.
||This panel allows you to include Xena testports in your test and to configure the behavior of these ports. It also allows you to include the ports in a service.
||This panel controls all aspects of the Service Configuration Tests, according to Y.1564 section 8.1.
||This panel controls all aspects of the Service Performance Tests, according to Y.1564 section 8.2.
||This panel allows you to define and configure bandwidth profiles used by all the UNIs in the configuration.
||This panel control all aspects of the reporting function.
|Result Data Grid
||This panel displays the test data in a grid view.
||This panel allows you to view a realtime plot of the current value for each of the four Service Acceptance Criteria (SAC) parameters (Frame Loss Ratio, Frame Transfer Delay, Frame Delay Variation and Availability).
Defined Services Tree View
All defined services will be shown in the Defined Services tree view in the left side of the application, as indicated in the image below.
The panel contains a small toolbar at the top with various tool icons for common operations (create/delete folder or service, add/remove UNI).
A checkbox is shown next to each service label. This checkbox determines whether the service will be included in a test. By default all new services are included in tests but if you temporarily want to disable a service you can just uncheck it.
The tree view contains two columns for each UNI entry which allow you to set the group and peer UNI for the UNI.
Each UNI can be a member of either the EAST or the WEST group. This setting is applicable only for Pair and Group test topologies.
When the test topology for the service is set to Pair you will need to specify the peer UNI for each UNI with this setting. A UNI can only be paired with a UNI from the opposite group.
This option is not available for EPL configurations since there are only two ports in an EPL and they are each other peer by definition.
The tree view allows you to select multiple entries at the same time using the standard Ctrl- and Shift-Click technique. Any configuration change performed when multiple entries are selected will be applied to all the selected entries.
The actual service configuration is handled by a trio of nested panels. Each panel will be displayed when the associated resource type is selected in the Defined Services tree view to the left.
It is possible to create a nested folder hierarchy to organize a large number of services. To create a subfolder under an existing folder just right-click the parent folder and select the Create Folder menu item. Enter a suitable name for the folder and press the OK button.
To delete a folder either right-click it and select the Delete Folder menu item or press the little Delete Folder icon in the small toolbar at the top of the Defined Services tree view.
To rename a folder you can just select it in the tree view and change the name in the configuration panel shown.
You can move a folder to a different parent folder by selecting the child folder with the mouse and drag it to the desired parent folder.
When you select a service in the Defined Services tree view the service configuration options are shown in the Service/UNI Configuration panel. These options are divided into three main groups.
Service Main Type Configuration
||A user-readable label string that can be used to identify the service. The string can take any value and can be modified later on.
||The type of service according to MEF 6.
|Is Virtual Service
||If not checked the service will not use service multiplexing on the UNI, i.e. the UNI will be wholly used/owned by the service.
If checked the service will use service multiplexing on the UNI based on the Customer VLAN tag (C-TAG). The UNI may thus be used by multiple services as long as each service uses a unique C-TAG value for the UNI.
Service Acceptance Criteria
The four Service Acceptance Criteria (SAC) parameters for the service is specified as part of the service definition. For each of the parameters you can also select whether to include it into the test using the checkboxes to the right.
The testflow characteristics determine how the Xena tester generates the necessary test traffic.
||Each UNI in the service is paired with another UNI on the same service and traffic will only flow between defined pairs. This requires the service to have an even number of UNIs.
||The UNIs are divided into two groups, EAST and WEST. Each member of one group will then send traffic to every member of the other group, depending on the Direction setting.
||Represents a true multipoint topology. Every UNI sends traffic to all other UNIs in the service. A Mesh is by nature always bidirectional.
Note: EPL services are considered to have a Pair topology by definition.
|East -> West
||Marks a unidirectional traffic pattern. Only UNIs in the EAST group will transmit data. Only UNIs in the WEST group will receive data.
|West -> East
||Marks a unidirectional traffic pattern. Only UNIs in the WEST group will transmit data. Only UNIs in the EAST group will receive data.
||The traffic flows both ways. Each UNI will both transmit and receive data.
Note: A Mesh topology will by definition always be bidirectional.
Moving Services between Folders
When a service is created it will normally be inserted as a child of the “All Services” root node in the Services View tree. If you create a service by using the right-click menu for a subfolder then service will however be created as a child of that folder.
When you select a UNI in the Defined Services tree view the UNI Configuration options are shown in the Service/UNI Configuration panel. It consists of three sub-panels in a tabbed view.
This subpanel allows you to specify the frame header for the UNI. In the top-left Frame Header Composition part you can select the protocol sections you want in your header. The Frame Editor in the lower part allow you to specify the value of selected fields the protocol header. The Frame Payload part to the right allow you to specify the content of the payload portion of the test frames.
Ingress Bandwidth Profiles
This subpanel allows you to specify the bandwidth profile(s) you want to use for your ingress UNI traffic.
You can either specify a per-UNI profile which will then be used for all traffic on the UNI. In this mode a single test stream will then be created to represent the profile.
Alternatively you can specify a number of per-CoS profiles for the UNI. You can specify between up to 8 bandwidth profiles, one for each CoS value available. This mode requires that your frame header is set to include a C-TAG VLAN.
In this mode a test stream is created for each CoS-profile mapping you specify. The Valkyrie1564 will override the PCP value in the C-TAG header for each stream with the CoS value.
You can also specify a DSCP value for each CoS-profile mapping. If your frame header includes an IP header the Valkyrie1564 application will override the DSCP value in the IP header with the specified value.
Egress Bandwidth Profiles
It is also possible to specify an egress bandwidth profile. Since the DUT/network may perform the traffic policing based on internally mapped priority values at the egress node it is not possible for the Valkyrie1564 to offer per-CoS bandwidth profiles for egress UNIs. You can specify a per-UNI profile which will then be used for all traffic streams going to that UNI.
The Physical Ports panel shows all defined Xena chassis and their ports. This panel will both display the current link and reservation state for each port and also enable configuration of port-specific settings.
The panel features a small tabbed view at the bottom which contains various port configuration parameters. These are explained below.
This panel contains physical port settings and also controls ARP and PING behavior of the port.
||Specifies the minimum gap between packets in the traffic generated for a port, expressed as a number of bytes.
||Specifies an optional speed reduction on the transmit side of the port, expressed as a ppm value.
|Enable PAUSE Mode:
||Controls whether the port responds to incoming PAUSE frames.
|Reply to ARP requests:
||Controls whether the port responds to incoming ARP requests.
Only relevant for an L3 traffic type.
|Reply to PING Requests:
||Controls whether the port responds to incoming PING requests.
Only relevant for an L3 traffic type.
|Send Gratuitous ARP:
||Controls whether the port will send Gratuitous ARP packets to defined gateways before starting the test.
Only relevant for an L3 traffic type.
This panel allow you to specify the various IP address settings for the port. It supports both IPv4 and IPv6 settings.
For IPv6 addresses you also have to enter the MAC address of the gateway. This is an interim requirement and will be removed in future releases.
Public IP Address
If a port is located behind a NAT firewall/router it may be necessary to provide the public IP address offered by the NAT firewall/router. The Valkyrie2544 will then perform an ARP request for the public IP address before starting the test, in order to avoid packet loss due to an initial ARP phase.
The real (internal) IP address of the port must still be configured in the main port tree as this may be used to send Gratuitous ARP packets from the port to the router before starting the test.
Remote Loop Endpoint
It is possible to specify that a port is connected to a remote loop by setting the Pair option to “Loop”. For layer-3 type traffic you must then specify the MAC address of the remote loop port to avoid excessive flooding. For layer-3 type traffic you must specify the IP address of the remote port so that the Xena tester can perform an ARP request for the MAC address.
Multiple Port Selection
It is possible to select multiple ports in the port tree by holding down either the <Shift> or the <Ctrl> key while selecting with the mouse. In this way the same option value can be applied to all selected ports. In the example shown below the Inter-Frame Gap value change will be applied to both the selected ports.
The Valkyrie1564 supports both the Configuration and Performance test types specified in Y.5164.
The Valkyrie1564 allows you to specify the frame sizes used for the test in two fundamentally different ways:
- Software Controlled Frame Sizes: Each test run uses a fixed frame size. You can specify multiple frame sizes using various algorithms, and this will thus result in multiple test runs, each with a different frame size..
- Hardware Controlled Frame Sizes: Each test run uses a set of frame sizes, generated by a specific algorithm and possibly bounded by a minimum and maximum frame size. As opposed to the software-controlled type the frame size will thus vary within each test run.
Available frame size algorithms are:
|Software Controlled Frame Sizes
||Uses the default set of frame sizes defined by IEEE: 64,128,256,512,1024,1280,and 1518 bytes.
||Allow you to specify a custom set of frame sizes.
||Allow you to specify an incrementing list, based on a start, end and step value.
|Hardware Controlled Frame Sizes
||Generates an incrementing list of sizes from a minimum to a maximum sizes, in steps of 1.
||Generates an alternating list on the form (min, max, min+1, max‐1, min+2, max‐2, etc.)
||Generates a list of random sizes between a minimum and maximum value.
||Generates a mixture of sizes between 56 and 1518 bytes, with an average of 464 bytes.
Service Configuration Tests
The Service Configuration Test procedure is configured in the Configuration Test panel. Here you can select which of the available teststeps you want to enable and also change other test settings.
The Valkyrie1564 supports the following sub-tests specified in Y.5164, section 8.1.2:
|CIR Validation Test
||Runs the simple CIR Validation Test, according to Y.1564, 8.1.2, A.1.
|CIR Step-Load Test
||Runs the CIR Step-Load Test, according to Y.1564, 8.1.2, A.2.
|EIR Configuration Test
||Runs the EIR Configuration Test, according to Y.1564, 8.1.2, B.2.
|Traffic Policing Test
||Runs the Traffic Policing Test, according to Y.1564, 8.1.2, C.2.
|CBS Configuration Test
||Runs a CBS Configuration Test. See implementation note below.
|EBS Configuration Test
||Runs a EBS Configuration Test. See implementation note below.
You will have to choose between running the CIR Validation Test or the CIR Step-Load Test as it does not make sense to run both. You can however select to run the Step-Load test if the simple Validation test fails.
The Valkyrie1564 does not implement the color-aware tests at the moment at it is somewhat unclear how the standard authors intends this to be performed.
CBS and EBS Configuration Tests
The Valkyrie1564 does not implement the CBS and EBS tests as specified in Y.1564, appendix I, as these are marked preliminary/experimental, and it is somewhat unclear how the standard authors intends these tests to be performed. Instead the test procedures specified in MEF 19, Test Cases 36 and 37 are used:
CBS Configuration Test
- The tester offers service frames with a average rate = CIR at the start of the test.
- At time T/2 (where T is the Step Duration parameter) the tester injects a single burst of back-to-back service frames with a total size greater than or equal to the CBS value.
- The test passes if the monitored values for FLR, FTD, and FDV all meet the service acceptance criteria specified for the service.
EBS Configuration Test
- The tester offers service frames with a average rate = CIR + EIR at the start of the test.
- At time T/2 (where T is the Step Duration parameter) the tester injects a single burst of back-to-back service frames with a total size greater than or equal to the EBS value.
- The test passes if the monitored values for FLR, FTD, and FDV all meet the service acceptance criteria specified for the service.
The following test parameters can be used to control the test:
||Normally a teststep should only be performed once. Setting the number of iterations > 1 will however make the Valkyrie1564 repeat each test step the specified number of times.
||The duration of each test step in seconds.
|Break Test On Fail:
||If checked the Valkyrie1564 will abort the tests when one of the teststeps fails, to enable debugging of the problem. If not checked the Valkyrie1564 will complete the whole test and report both passed and failed tests.
||The start rate in percent of CIR used when running the CIR Step-Load test.
||The step rate in percent of CIR used when running the CIR Step-Load test.
||The “M” factor mentioned in Y.1564, section 8.1.2, C.2.
||Controls how the Frame Transfer Delay (aka. “latency”) is measured by the Xena tester.
Layer-3 Address Refresh
If the frame header for one or more of the UNIs include IP the Valkyrie1564 will resolve gateway IP addresses to MAC addresses before the test starts using either ARP (IPv4) or NDP (IPv6). The Valkyrie1564 will also ensure that the IP gateways learns the MAC addresses of the tester ports to avoid frame loss due to mid-test ARP/NDP resolution.
For long-running tests the ARP/NDP cache entries in the IP gateways may time out before the test completes. To avoid this the Valkyrie1564 will periodically issues ARP/NDP “keep-alive” requests to maintain the cache entries. You can control the period with which this is performed in the L3 Address Refresh section. You can also disable the refresh function completely.
Service Performance Tests
The Service Performance Test procedure is configured in the Performance Test panel. Here you can select which of the available teststeps you want to enable and also change other test settings.
||Determines the length of the test period. Y.1564 defines mandatory periods of 15 minutes, 2 hours and 24 hours, so these are directly available. But it is also possible to specify a custom period in a hours:minutes:second format. It is also possible to specify an “unbounded” period, which essentially means that the test will run until it is stopped manually.
|Frame Loss Ratio for SES:
||Define the Frame Loss Ratio threshold necessary to declare a Severe Errored Second. This is used when calculating the availability of the services according to Y.1563 section 9.2. Y.1563 section 9.1 proposes a default ratio of 0.5 but you can change this value if necessary.
The description for the remaining test parameters are identical to the similar parameters for the Configuration Test.
The Bandwidth Profiles panel enables you to define and configure the bandwidth profiles used for the UNIs. The Valkyrie1564 uses a separate definition of the bandwidth profiles to mimic the real-life service definitions.
This panel contains a number of options that affect the way reports are generated for the test.
This section contains a number of options that can help identify the test context.
- Customer Name: The name of the customer for which the test is performed.
- Customer Service/Access Id: These two options allow you to provide details about the network circuits you are testing.
- Comments: Allows you to provide any multi-line comments for the test configuration.
This section contains options that affect the way reports are generated.
- Report Filename Prefix: Specifies the prefix for the report filename.
- Append Timestamp to Filename: If checked a timestamp on the form <YYYYMMDD-HHMMSS> is appended to the filename prefix.
- Include Graphs in Report: If checked the report will include bar charts showing the test results. This is only applicable for PDF type reports.
This section allows you to select which types of reports will be generated. You can enable several types. The generated report files will be given a file extension that matches the selected type, i.e. “.pdf“ for PDF files and so forth.
XML Report Specification
You can find the specification for the XML Report here.
Selecting Services for Test
There are two ways to include a service in the Configuration or Performance tests. The main method is to use the checkbox next to the service label in the Defined Services tree view to include or exclude a service from the tests. When a service is created it will by default be enabled for test, but it is possible to exclude it afterwards.
You can also right-click on a service in the tree view and select to run either the Configuration or Performance tests for this service only.
Scheduling Service Tests
Although you can select multiple services for a test run the services are scheduled according to Y.1564, depending of the test type.
For Configuration Tests each service is tested independently. This means that although you can select multiple services for the test each service is tested on its own. In other words, the selected services are tested in serial.
For Performance Tests all services are however tested in parallel, as specified in Y.1564.
Result Display and Monitoring
The application features two panels at the bottom which can display the progress and result of the tests. The Result Data Grid displays the result data in a tabular way. The Result Plot panel displays a realtime plot of the four SAC parameters.
Result Data Grid
The Result Grid display is shown below. The results for each service is shown in a separate tab in the panel. The left-most columns show the total results for all UNIs in the service.The right-most columns show the results for each UNI-pair. If the UNI bandwidth profiles has been assigned per-CoS the grid will display a set of results fro each UNI-pair/CoS set.
The panel shows both the final results and the running progress results. The final results are shown in a normal font whereas the running progress results are shown with an italics font.
The Result Plot contains a tabbed view that displays a running plot of the 4 SAC parameters Loss Ratio, Transfer Delay, Delay Variation and Availability. By default the plot view displays data from the last minute, but this can be changed in the control at the bottom of the panel.
Command Line Invocation
The separate Valkyrie1564run executable can be used to execute a Valkyrie1564 configuration file from the command line or from a script environment. Refer to the separate ValkyrieRun manual for details.
The Valkyrie1564 application features a number of application-level settings which are valid for all test configurations. These can be accessed in the Options menu item.
- Set Username: Allows you to specify the username used when logging in to the Xena testers. The default value will be your Windows username (shortened to 8 characters if longer).
- Ask to Relinquish Ports: If checked the Valkyrie1564 will prompt you when you try to use a port which is currently reserved by someone else. If unchecked the Valkyrie2544 will unceremoniously relinquish the port and reserve it in your name.
- Set Reports Path: Allows you to specify a different location for the generated reports. The default location is <ProgramData>XenaNetworksXena1564Reports. The location of<ProgramData> varies between versions of Windows but can be obtained by inspecting the environment variable “ProgramData”.
- Set Tester Company Name: Allows you to set the name of the company performing the tests. The default value is “Xena Networks ApS”
- Set Tester Company Logo File: Allows you to specify the path to an image file which will be used as the logo in the reports. The default logo is the Xena Networks logo. The image can be in PNG or JPG format. It should have the approximate dimensions of 150 x 60 pixels and be set to a DPI of 96.
- Open Report in Associated Viewer: If checked the Valkyrie1564 will attempt to open the generated report in the associated viewer. This is currently only applicable for PDF reports. This requires that a suitable PDF viewer (such as the Adobe Acrobat Reader) is installed on your PC.
Customizing the UI Layout
Navigating Tabbed Panels
The application offers several ways to navigate between the function panels in the main tabbed view:
- Select the panel by clicking on its tab selector.
- Use the <Ctrl-Tab> key combination to cycle between the panels.
- Use the View menu to select (and possibly display) a panel.
Panel Display Options
A function panel can be in one of three states, displayed, hidden and closed. When a panel is displayed it is shown at all times. When a panel is hidden it collapses when not in focus, only leaving its tab selector visible. When a panel is closed no part of it will be visible. The default state of the panels in the main tabbed view is displayed whereas the default state for the two result controls at the bottom of the main view is hidden.
To temporarily display a hidden panel you can just let the mouse hover over its tab selector. When the panel loses focus (typically because some other panel is selected) it will auto-hide itself again.
You can change between the displayed and hidden states for a panel by clicking the little “paper tack” icon in the upper right corner of the panel, as indicated in the image below. When the tack is positioned vertically the panel is “pinned”, i.e. displayed at all times. When the tack is positioned horizontally the panel will auto-hide itself when it looses focus.
|“Tack” icon that toggles auto-hide behavior
Panels in the main tabbed view are always in the displayed state and cannot be hidden. If a panel is moved to another tab container it can be hidden if desired.
You can close a panel by clicking the little ‘x’ in the right corner of the panel. The panel will then not be shown until you select it in the View menu.