Universal decode unit testing

Universal Decode unit tests are based on the Lunit unit test framework, one of the standard Lua extensions.

The Lunit framework provides you with the set of functions you can use to test your Universal Decode scripts. Refer to Lunit project page for details. The basic idea for Universal Decode unit tests is to supplement your tests with the snippets of payload generated by the traffic of your interest, and then test them against the parse_request and parse_response functions of your script.

Getting the test payload snippets

The easiest way to to extract the payload from a tcpdump  in the form you can paste it into your test is to use Wireshark. Open a tcpdump, r ight-click the packets list, select Follow > TCP Stream  and choose to show the data as C Arrays (Show data as list).This way, the payload is presented as a hex array, ready to copy and paste into your lua test script.

Running your tests

Providing your Eclipse workspace is set up properly, you can run the test directly in Eclipse.

Eclipse test environment
Eclipse test environment

If you prefer to run the test in the Linux terminal instead of Eclipse, navigate to the directory with the test script and run the command as in the example below.

lunit sampleTest.lua

Writing the test script

When reading this short tutorial, have the sampleTest.lua  file handy for your reference.

Start the unit test script by referring your actual Lua script, the Universal Decode stats reference and the Lunit framework:

Stats = require 'stats'
local sampleScript = require 'sampleScript'
local lunit = require 'lunitx'

Lunit uses the lua-5.1 module system. A testcase is an arbitrarily named module marked with lunit.testcase as a testcase. In our example, you need to declare the module using the following syntax:

  module( "sampleScript_test", lunit.testcase, package.seeall)

Finally, you can start adding the tests. The tests itself are functions with names that must begin or end with the test string. The function names are case insensitive. For example:

 function test_request()
          -- the test contents goes here
        end

The information above refers to the Lua unit tests in general. Now, we are about to focus on Universal Decode related practices and hints. As mentioned before, you should be able to extract snippets of payload from your traffic, representing the type of information and structures you want to extract using the Universal Decode script. Paste the textual payload snippet as the comma separated array as in the example below:

 local payload_arr = {
    'GET /videoplayback/aa/bbb?algorithm=throttle-factor&burst=40s HTTP/1.1\r\n',
    'Host: r6---sn-oxup5-3ufe.c.youtube.com\r\n',
    'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0\r\n',
    'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n',
    'Accept-Language: en-US,en;q=0.5\r\n',
    'Accept-Encoding: gzip, deflate\r\n',
    'Referer: http://s.ytimg.com/yts/swfbin/watch_as3-vfldOoVEA.swf\r\n',
    'Cookie: VISITOR_INFO1_LIVE=6aStOq9K_eY; PREF=f4=210020&al=en&fv=11.2.202&f1=50000000; ACTIVITY=1373534604727\r\n',
    'Connection: keep-alive\r\n',
    'Authorization: Basic c3VwZXJ1c2VyOg==  \r\n',  
    '\r\n'
  } 

To improve the test performance, concatenate the table using the following function:

 local payload_str = table.concat(payload_arr, '')

In case of the binary protocols, add the payload in the following way:

  local payload_arr = {
  0x51, 0xac, 0xed, 0x00, 0x05, 0x77, 0x0f, 0x01, 0x66, 0x28, 0x5b, 0x87, 0x00, 0x00, 0x01, 0x3e,
  0x88, 0x88, 0xb3, 0x95, 0x80, 0x13, 0x74, 0x00, 0x08, 0x47, 0x72, 0x65, 0x65, 0x74, 0x69, 0x6e,
  0x67
  }

And optimize it with:

 local payload_str = string.char(unpack(payload_arr))

Declare the stats object, so that unit test is able to execute your Universal Decode script functions:

 local obj = Stats()

Specify the function you want assert in the test. Precede the function name with the script name as referred in the first lines of the test:

 local parse_status = sampleScript.parse_request(payload_str, obj)

Define the test output and asserts that will test your parse_request functions against the sample payload.

See the example below:

 print('Request:')
  print('\tparse_status:', parse_status)
  print('\top name:', '"'..obj.operation_name..'"')
--  assert(obj.operation_name == "r6---sn-oxup5-3ufe.c.youtube.com/videoplayback/aa/bbb")
  assert(obj.operation_name == "r6---sn-oxup5-3ufe.c.youtube.com")
  print("\tos version:", '"' .. or_nil_str(obj.os_name) .. '"')
  assert(obj.os_name == "Ubuntu", message(obj.os_name, 'Ubuntu'))
  print("\tbrowser:", '"' .. or_nil_str(obj.browser) ..'"')
  assert(obj.browser == "Firefox", message(obj.browser, 'Firefox')) 
  print("\tbrowser version:", '"'.. or_nil_str(obj.browser_version) ..'"')
  assert(obj.browser_version == "22.0", message(obj.browser_version, '22.0'))
  print("\thardware:", '"'.. or_nil_str(obj.hardware) ..'"')
  assert(obj.hardware == "Linux", message(obj.hardware, 'Linux'))

And here's the output of the test above.

 Loaded testsuite with 1 tests in 1 testcases.

Request:
	parse_status:	0
	op name:	"r6---sn-oxup5-3ufe.c.youtube.com"
	os version:	"Ubuntu"
	browser:	"Firefox"
	browser version:	"22.0"
	hardware:	"Linux"
    .

5 Assertions checked.

Testsuite finished (1 passed, 0 failed, 0 errors).

Remember that for the unit tests to work, your Universal Decode script must end with the following lines:

local the_module = {}
the_module.parse_request = parse_request
the_module.parse_response = parse_response
return the_module