Packet Hex Dump

2012/04/15

Extracting Information From a Hexadecimal Packet Dump

Sometimes we may obtain a hexadecimal dump of some part of a packet, and we may need to get the data into some format that allows us to do something with it, like analyzing it, rewriting or replaying it, etc. Network Expect has some built-in capabilities to handle hexadecimal packet dumps that are similar in spirit to Wireshark’s text2pcap utility. This page provides a couple of examples of how to import hexadecimal packet dumps and make use of them.

Deciphering an OSPF Message

The first example shows how to dissect an OSPF message. When we start, we only have a hexadecimal packet dump and a brief description that tells us that the packet is an OSPF message. By looking at the packet dump below it seems clear that it comes from tcpdump, but we could have received it via email, we may have seen it in some C or Python source code, etc.

12:27:53.704034 vlan 1, p 6, IP gw-ext.chapus.net > OSPF-ALL.MCAST.NET: OSPFv2, Hello, length 60
        0x0000:  c001 0800 45c0 0050 ea14 0000 0159 d86f  ....E..P.....Y.o
        0x0010:  0a0a 0c02 e000 0005 0201 0030 0a0a 0c02  ...........0....
        0x0020:  0000 0000 946c 0000 0000 0000 0000 0000  .....l..........
        0x0030:  ffff ff00 000a 1201 0000 0028 0a0a 0c02  ...........(....
        0x0040:  0a0a 0c01 0a0a 0c01 fff6 0003 0001 0004  ................
        0x0050:  0000 0001                                ....

Note that tcpdump is telling us that this packet has a 802.1q tag (the “vlan 1, p 6” part) that precedes an IP packet (0x45, 0xc0, etc.)

Our objective is to dissect this packet dump in Network Expect. Note that we could easily use Wireshark’s text2pcap to convert the dump to PCAP format and then open it in Wireshark, but the assumption is that we may have to do something more once we have the data in a format that Network Expect can use.

The first thing to do after we have the hexadecimal packet dump is to import the data into Network Expect. The best Network Expect data object to store the imported data in is a BArray object, which is a custom Tcl object that Network Expect implements to store binary data. Network Expect uses the barray command to manipulate BArrays, and this command has a hex-import option that we need to make use of. To import the above hexadecimal packet dump we use the barray command in this way:

netexpect> set dot1q [barray hex-import -ignore-prefix 1 -ignore-after-column 58 {
>        0x0000:  c001 0800 45c0 0050 ea14 0000 0159 d86f  ....E..P.....Y.o
>        0x0010:  0a0a 0c02 e000 0005 0201 0030 0a0a 0c02  ...........0....
>        0x0020:  0000 0000 946c 0000 0000 0000 0000 0000  .....l..........
>        0x0030:  ffff ff00 000a 1201 0000 0028 0a0a 0c02  ...........(....
>        0x0040:  0a0a 0c01 0a0a 0c01 fff6 0003 0001 0004  ................
>        0x0050:  0000 0001                                ....
>} ]
\xc0\x01\x08\x00\x45\xc0\x00\x50\xea\x14\x00\x00\x01\x59\xd8\x6f\x0a\x0a\x0c\x02\xe0\x00\x00\x05\x02\x01\x00\x30\x0a\x0a\x0c\x02\x00\x00\x00\x00\x94\x6c\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\x00\x00\x0a\x12\x01\x00\x00\x00\x28\x0a\x0a\x0c\x02\x0a\x0a\x0c\x01\x0a\x0a\x0c\x01\xff\xf6\x00\x03\x00\x01\x00\x04\x00\x00\x00\x01

Note the use of the -ignore-prefix and the -ignore-after-column options: the former is needed because the hexadecimal dump contains an offset field that we do not need, and the latter is needed because the dump contains an ASCII representation on the right of the data. -ignore-prefix 1 causes one field (word) at the beginning of each line to be ignored, and -ignore-after-column causes anything after character 58 in any input line to be ignored. The number of “prefix words” to ignore is easily determined by visual inspection. The column number at which we start to ignore input can be determined by pasting a sample input line into your favorite text editor, moving the cursor to the column after which all data needs to be ignored, i.e. right before the ASCII representation of the data, and then looking at your editor’s column number.

The above command was obviously executed interactively, i.e. not from a script (notice the “netexpect>” prompt and the ‘>’ line continuation characters). The hexadecimal dump was copied and pasted into the terminal window where Network Expect was being run.

The string “\xc0\x01\x08\x00\x45\xc0\x00…” is now stored in the variable $dot1q, which has a BArray type:

netexpect> set dot1q
\xc0\x01\x08\x00\x45\xc0\x00\x50\xea\x14\x00\x00\x01\x59\xd8\x6f\x0a\x0a\x0c\x02\xe0\x00\x00\x05\x02\x01\x00\x30\x0a\x0a\x0c\x02\x00\x00\x00\x00\x94\x6c\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\x00\x00\x0a\x12\x01\x00\x00\x00\x28\x0a\x0a\x0c\x02\x0a\x0a\x0c\x01\x0a\x0a\x0c\x01\xff\xf6\x00\x03\x00\x01\x00\x04\x00\x00\x00\x01

We can manipulate this variable using the barray Network Expect command. For example:

netexpect> barray slice dot1q 0l4
\xc0\x01\x08\x00

(This uses the barray command to extract four bytes starting at offset 0 of the $dot1q BArray variable.)

Once we have imported the dump into Network Expect our next task is to convert it to a Network Expect Packet object so we can dissect it. We do this using Network Expect’s packet command:

netexpect> set pkt [packet new -y EN10MB ether(type = 0x8100)/'$dot1q']
  10.10.12.2 -> 224.0.0.5    OSPF Hello Packet

Note a couple of important points:

We used the -y switch of the packet new command to specify the data link type -- we do this because the packet dump starts with a 802.1q tag.

Because the packet dump starts with a 802.1q tag, the PDU definition in the packet new command uses "ether(type = 0x8100)". This gets prepended to the actual dump, which is stored in the $dot1q variable. 

After using the packet new command and putting its result in the $pkt variable, $pkt now has the data in Packet format and we can use the packet command to dissect it:

netexpect> packet show pkt
---[ frame ]---
  frame.time: "Aug 15, 2010 12:43:41.339683000 EDT"
  frame.time_epoch: 1281890621.339683000
  frame.time_delta: 0.2147483647
  frame.time_delta_displayed: 0.2147483647
  frame.time_relative: 0.2147483647
  frame.number: 0
  frame.len: 98
  frame.cap_len: 98
  frame.marked: 0
  frame.ignored: 0
  frame.protocols: eth:vlan:ip:ospf
---[ eth ]---
  eth.dst: 00:00:00:00:00:00
    eth.addr: 00:00:00:00:00:00
    eth.ig: 0
    eth.lg: 0
  eth.src: 00:00:00:00:00:00
    eth.addr: 00:00:00:00:00:00
    eth.ig: 0
    eth.lg: 0
  eth.type: 33024
---[ vlan ]---
  vlan.priority: 6
  vlan.cfi: 0
  vlan.id: 1
  vlan.etype: 2048
---[ ip ]---
  ip.version: 4
  ip.hdr_len: 20
  ip.tos: 192
    ip.tos.precedence: 6
    ip.tos.delay: 0
    ip.tos.throughput: 0
    ip.tos.reliability: 0
    ip.tos.cost: 0
    ip.flags: 0
      ip.flags.rb: 0
      ip.flags.df: 0
      ip.flags.mf: 0
  ip.len: 80
  ip.id: 59924
  ip.frag_offset: 0
  ip.ttl: 1
  ip.proto: 89
  ip.checksum: 55407
    ip.checksum_good: 1
    ip.checksum_bad: 0
  ip.src: 10.10.12.2
  ip.addr: 10.10.12.2
  ip.src_host: 10.10.12.2
  ip.host: 10.10.12.2
  ip.dst: 224.0.0.5
  ip.addr: 224.0.0.5
  ip.dst_host: 224.0.0.5
  ip.host: 224.0.0.5
---[ ospf ]---
  Text label: OSPF Header
    Text label: OSPF Version: 2
    ospf.msg: 1
    ospf.msg.hello: 1
    Text label: Packet Length: 48
    ospf.srcrouter: 10.10.12.2
    Text label: Area ID: 0.0.0.0 (Backbone)
    Text label: Packet Checksum: 0x946c [correct]
    Text label: Auth Type: Null
    Text label: Auth Data (none)
  Text label: OSPF Hello Packet
    Text label: Network Mask: 255.255.255.0
    Text label: Hello Interval: 10 seconds
    ospf.v2.options: 18
      ospf.v2.options.dn: 0
      ospf.v2.options.o: 0
      ospf.v2.options.dc: 0
      ospf.v2.options.l: 1
      ospf.v2.options.np: 0
      ospf.v2.options.mc: 0
      ospf.v2.options.e: 1
      ospf.v2.options.mt: 0
    Text label: Router Priority: 1
    Text label: Router Dead Interval: 40 seconds
    Text label: Designated Router: 10.10.12.2
    Text label: Backup Designated Router: 10.10.12.1
    Text label: Active Neighbor: 10.10.12.1
  Text label: OSPF LLS Data Block
    Text label: Checksum: 0xfff6
    Text label: LLS Data Length: 12 bytes
    Text label: Extended options TLV
      Text label: Type: 1
      Text label: Length: 4
      ospf.lls.ext.options: 1
        ospf.lls.ext.options.rs: 0
        ospf.lls.ext.options.lr: 1

The packet dump could also be replayed. Here is one possible way of accomplishing this:

netexpect> send_network -o hex data(data = [barray slice dot1q 4:])
Packet size: 80 bytes

00000000  45c0 0050 ea14 0000 0159 d86f 0a0a 0c02 E..P.....Y.o....
00000010  e000 0005 0201 0030 0a0a 0c02 0000 0000 .......0........
00000020  946c 0000 0000 0000 0000 0000 ffff ff00 .l..............
00000030  000a 1201 0000 0028 0a0a 0c02 0a0a 0c01 .......(........
00000040  0a0a 0c01 fff6 0003 0001 0004 0000 0001 ................
00000050                                          

!
1

Note that here we are ignoring the first four bytes of the original packet dump (the 802.1q tag) and injecting the IP packet. In this case we do not need to specify an Ethernet header.

BGP UPDATE Message Affects Internet Routing Stability

On Friday August 27, 2010 around 8:41 UTC, an experiment involving BGP was conducted by RIPE NCC and researchers at Duke University. The experiment involved announcing a prefix with an unknown transitive attribute. Due to a bug in a router vendor’s BGP implementation, the experiment caused BGP connection resets that affected the stability of the Internet during the duration of the experiment. The issue has been explained in detail here and here. The first link actually contains an exact description of the attribute that caused the problem.

This is relevant to this example because Cisco routers that reset BGP connections due to a bug in the peering router’s BGP implementation logged syslog messages like this:

Aug 27 10:42:45.455: %BGP-3-NOTIFICATION: sent to neighbor 172.16.63.133 3/1 (update malformed) 188 bytes F0630BB8 00000000 00000000 00000000 00
Aug 27 10:42:45.455: BGP: 81.46.63.133 Bad attributes FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 00F4 0200 0000 D940 0101 0040 0208 0203 0D18 329C 316E 4003 0451 2E3F 85C0 0804 0D18 0078 F063 0BB8 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 185D AF90

Now, the question is: how do we go about decoding this hexadecimal dump to understand the experiment RIPE and Duke University performed on August 27, 2010? Well, we could use Network Expect’s hexadecimal import and packet dissection capabilities:

First, we notice that there are 16 bytes at the beginning of the hexadecimal dump that have the same value 0xff. People familiar with BGP and RFC 1771 will notice that this is likely the BGP marker in the BGP header. So with that in mind we import this hexadecimal dump into a Tcl BArray variable using the barray hex-import command:

netexpect> set bgpmsg [barray hex-import {FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF 00F4 0200 0000 D940 0101 0040 0208 0203 0D18 329C 316E 4003 0451 2E3F 85C0 0804 0D18 0078 F063 0BB8 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 185D AF90}]

After importing the hexadecimal dump into a Tcl BArray variable, the variable itself looks like a long character string when the variable is used in a string context:

netexpect> set bgpmsg
\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\xf4\x02\x00\x00\x00\xd9\x40\x01\x01\x00\x40\x02\x08\x02\x03\x0d\x18\x32\x9c\x31\x6e\x40\x03\x04\x51\x2e\x3f\x85\xc0\x08\x04\x0d\x18\x00\x78\xf0\x63\x0b\xb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x18\x5d\xaf\x90

We then can create a Packet Tcl variable using the packet new command:

netexpect> set pkt [packet new ip(dst = 1.1.1.1)/tcp(dst = 179)/'$bgpmsg']

Note how we prefix the actual BGP message with layer 3 (IP) and layer 4 (TCP) headers, and the destination port for the TCP segment is 179, the BGP port.

After the above packet new command is run, the variable $pkt contains the BGP message that we are interested in dissecting. The Tcl string representation of the packet variable is actually a short description of the packet:

netexpect> set pkt
10.82.244.122 -> 1.1.1.1      BGP [TCP Retransmission] UPDATE Message

To actually dissect the packet we use the packet show command:

netexpect> packet show pkt
---[ frame ]---
  frame.time: "Aug 31, 2010 21:58:04.218379000 EDT"
  frame.time_epoch: 1283306284.218379000
  frame.time_delta: 0.2147483647
  frame.time_delta_displayed: 0.2147483647
  frame.time_relative: 0.2147483647
  frame.number: 0
  frame.len: 284
  frame.cap_len: 284
  frame.marked: 0
  frame.ignored: 0
  frame.protocols: raw:ip:tcp:bgp
---[ raw ]---
  Text label: No link information available
---[ ip ]---
  ip.version: 4
  ip.hdr_len: 20
  ip.tos: 0
    ip.tos.precedence: 0
    ip.tos.delay: 0
    ip.tos.throughput: 0
    ip.tos.reliability: 0
    ip.tos.cost: 0
  ip.len: 284
  ip.id: 256
  ip.flags: 0
    ip.flags.rb: 0
    ip.flags.df: 0
    ip.flags.mf: 0
  ip.frag_offset: 0
  ip.ttl: 64
  ip.proto: 6
  ip.checksum: 30734
    ip.checksum_good: 1
    ip.checksum_bad: 0
  ip.src: 10.82.244.122
  ip.addr: 10.82.244.122
  ip.src_host: 10.82.244.122
  ip.host: 10.82.244.122
  ip.dst: 1.1.1.1
  ip.addr: 1.1.1.1
  ip.dst_host: 1.1.1.1
  ip.host: 1.1.1.1
---[ tcp ]---
  tcp.srcport: 1073
  tcp.dstport: 179
  tcp.port: 1073
  tcp.port: 179
  tcp.stream: 0
  tcp.len: 244
  tcp.seq: 0
  tcp.nxtseq: 244
  tcp.hdr_len: 20
  tcp.flags: 0
    tcp.flags.cwr: 0
    tcp.flags.ecn: 0
    tcp.flags.urg: 0
    tcp.flags.ack: 0
    tcp.flags.push: 0
    tcp.flags.reset: 0
    tcp.flags.syn: 0
    tcp.flags.fin: 0
  tcp.window_size: 8192
  tcp.checksum: 25769
    tcp.checksum_good: 1
    tcp.checksum_bad: 0
  Text label: SEQ/ACK analysis
    tcp.analysis.bytes_in_flight: 244
      tcp.analysis.rto: 8.975946000
      tcp.analysis.rto_frame: 0
---[ bgp ]---
  Text label: UPDATE Message
    Text label: Marker: 16 bytes
    Text label: Length: 244 bytes
    bgp.type: 2
    Text label: Unfeasible routes length: 0 bytes
    Text label: Total path attribute length: 217 bytes
    Text label: Path attributes
      Text label: ORIGIN: IGP (4 bytes)
        Text label: Flags: 0x40 (Well-known, Transitive, Complete)
          Text label: 0... .... = Well-known
          Text label: .1.. .... = Transitive
          Text label: ..0. .... = Complete
          Text label: ...0 .... = Regular length
        Text label: Type code: ORIGIN (1)
        Text label: Length: 1 byte
        bgp.origin: 0
      Text label: AS_PATH: 3352 12956 12654 (11 bytes)
        Text label: Flags: 0x40 (Well-known, Transitive, Complete)
          Text label: 0... .... = Well-known
          Text label: .1.. .... = Transitive
          Text label: ..0. .... = Complete
          Text label: ...0 .... = Regular length
        Text label: Type code: AS_PATH (2)
        Text label: Length: 8 bytes
        Text label: AS path: 3352 12956 12654
          Text label: AS path segment: 3352 12956 12654
            Text label: Path segment type: AS_SEQUENCE (2)
            Text label: Path segment length: 3 ASs
            Text label: Path segment value: 3352 12956 12654
            bgp.as_path: 3352
            bgp.as_path: 12956
            bgp.as_path: 12654
      Text label: NEXT_HOP: 81.46.63.133 (7 bytes)
        Text label: Flags: 0x40 (Well-known, Transitive, Complete)
          Text label: 0... .... = Well-known
          Text label: .1.. .... = Transitive
          Text label: ..0. .... = Complete
          Text label: ...0 .... = Regular length
        Text label: Type code: NEXT_HOP (3)
        Text label: Length: 4 bytes
        bgp.next_hop: 81.46.63.133
      Text label: COMMUNITIES: 3352:120 (7 bytes)
        Text label: Flags: 0xc0 (Optional, Transitive, Complete)
          Text label: 1... .... = Optional
          Text label: .1.. .... = Transitive
          Text label: ..0. .... = Complete
          Text label: ...0 .... = Regular length
        Text label: Type code: COMMUNITIES (8)
        Text label: Length: 4 bytes
        Text label: Communities: 3352:120
          Text label: Community: 3352:120
            bgp.community_as: 3352
            bgp.community_value: 120
      Text label: Unknown (3004 bytes)
        Text label: Flags: 0xf0 (Optional, Transitive, Partial, Extended Length)
          Text label: 1... .... = Optional
          Text label: .1.. .... = Transitive
          Text label: ..1. .... = Partial
          Text label: ...1 .... = Extended length
        Text label: Type code: Unknown (99)
        Text label: Length: 3000 bytes
        Text label: Unknown (3000 bytes)
    Text label: Network layer reachability information: 4 bytes
      Text label: 93.175.144.0/24
        Text label: NLRI prefix length: 24
        bgp.nlri_prefix: 93.175.144.0

And that’s all – by looking at the packet dissection we can know the details about the BGP update message that caused the problem, i.e. the Autonomous System the update came from (12654), the prefix (93.175.144.0/24), the path attribute (type 99), the size (3000 bytes), etc. Note that this confirms the description of the problem at http://seclists.org/nanog/2010/Aug/774, which states:

“The experimental attribute was part of an experiment conducted in collaboration with a group from Duke University. This involved announcing a large (3000 bytes) optional transitive attribute, using a modified version of Quagga. The attribute used type code 99. The data consisted of zeros. We used the prefix 93.175.144.0/24 for this and announced from AS 12654 on AMS-IX, NL-IX and GN-IX to all our peers.”

For the curious, the impacted BGP implementation was on Cisco IOS XR (Cisco IOS was not affected) and Cisco published a Cisco Security Advisory on the same day: http://www.cisco.com/warp/public/707/cisco-sa-20100827-bgp.shtml.

Conclusion

These are simple examples, but they show how to quickly analyze hexadecimal packet dumps in a very short time and using very few commands and without the need to write specialized tools.