SCAPY Full Duplex Stream Reassembly

 

I recently had someone ask me how you can have scapy reassemble full duplex packets for you. That is what Wireshark does when you ask it to “Follow TCP Stream”. In SANS SEC573: Automating Information Security with Python we discuss how to use scapy’s native session reassembly capabilities, but its default behavior is to reassemble unidirectional streams. In other words, two separate sessions are created. One for traffic that flows from Host A to Host B and another for the traffic in the same session that flows form Host B to Host A. To use scapy’s native session reassembly you call a packet list’s .sessions() method which returns a Python dictionary of followed streams. They keys of that dictionary contain a string indicating who is communicating with who in the format "Protocol SourceIP:SourcePort > DestinationIP:DestinationPort". Its value in the dictionary is a list of all the packets that are part of that stream.

duplex_blogimage1

 

In this example you can tell by looking at the keys that there are two sessions. In the course we go into more detail about how to reassemble these packets to extract payloads so you can look for useful pieces of attack signatures. But, the question posed to me was “How can I do this with full duplex streams combining those two into a single session like Wireshark?” There are a couple of ways you could do this. One option is just to combine the two sets of packets.

duplex_blogimage2

 

This works well for a few packets, but at scale can be problematic. For example, if you step through a large file you have to keep track of which streams you already combined so that when you reach the 2nd half of the conversation you don’t process the same stream a second time. A better option is to pass the scapy PacketList session() method a function that tells it how to reassemble the packets in full duplex. The function will take in a single packet and returns a string that contains the relevant bits of data that uniquely identify the steam. Every packet in your list of packets that has the same string will be automatically grouped together into a stream by the sessions() method. This behaves sort of like a key function for Python’s sort capability. For our packets to be full duplex we need the string to be identical for both HOST A to HOST B and HOST B to HOST A communications. You can accomplish this by just sorting a list of the attributes that make our stream unique.  Here is a function called full_duplex() that will reassemble all of the same protocols that the default session assembly process currently supports but will do it with bi-directional streams.

duplex_blogimage3

 

To use the function pass the full_duplex function to the sessions() method and it will use it to reassemble your packets.

duplex_blogimage4

 

Here is an example of trying the full_duplex() function on a larger pcap file with multiple sessions.

duplex_blogimage5

 

You can see that without full_duplex you have 182 unidirectional streams and with full_duplex you have 91 bidirectional streams. Now, using various techniques you can reassemble the payload from the full duplex streams and search for evil, forensics data or useful pieces of information.  For more information on this and other techniques check out SANS SEC573: Automating Information Security with Python.

The Scapy Packet reassembly:

https://gist.github.com/MarkBaggett/d8933453f431c111169158ce7f4e2222#file-scapy_helper-py

  • Mark Baggett

 

Upcoming SANS Special Event – 2018 Holiday Hack Challenge

KringleCon

SANS Holiday Hack Challenge – KringleCon 2018

  • Free SANS Online Capture-the-Flag Challenge
  • Our annual gift to the entire Information Security Industry
  • Designed for novice to advanced InfoSec professionals
  • Fun for the whole family!!
  • Build and hone your skills in a fun and festive roleplaying like video game, by the makers of SANS NetWars
  • Learn more: www.kringlecon.com
  • Play previous versions from free 24/7/365: www.holidayhackchallenge.com

Player Feedback!

  • “On to level 4 of the #holidayhackchallenge. Thanks again @edskoudis / @SANSPenTest team.” – @mikehodges
  • “#SANSHolidayHack Confession – I have never used python or scapy before. I got started with both today because of this game! Yay!” – @tww2b
  • “Happiness is watching my 12 yo meet @edskoudis at the end of #SANSHolidayHack quest. Now the gnomes #ProudHackerPapa” – @dnlongen
kringle_02

SQLMAP Tamper Scripts for The Win

 

During a recent penetration test BURP Suite identified some blind SQL Injection vulnerabilities in a target website. Pointing SQLMAP at the website showed us no love and simply said it was unable to exploit the website. I had mentioned the SQLi issues to the customer and he said that previous penetration testers said they were unexploitable. We decided to take a closer look anyway. The URLs for the website looked rather odd.  I can’t talk specifically about the website in question, but the URIs looked something like this:

 

 

http://www.example.tgt/website.php?QnnyBZ4_ZB6qvm=xxxTcTc&k3mK4_ZQ6v=6V9A&UQK4_ZQ6v=qVllgrr

SQLMAP_Tamper_Script01

 

You’ll notice that the field names (underlined in RED) have very strange names.   At first I thought that these were just weird field names. Maybe the developer has some codenames for fields that I just didn’t understand. But then I noticed that the values (underlined in BLUE) were also very odd. None of the information on the URL made any sense to me. I grabbed a coworker and we spent some time trying to figure out what kind of weird encoding was being used. The web application had some useful functionality that make the translation pretty easy to figure out. If we put "AAAAAAAAA" into the ACCOUNT NUMBER field in the websites search page we saw that it redirected us to a web page with a URI  containing 'QnnyBZ4_ZB6qvm=QQQQQQQQQ'. When we searched for an ACCOUNT NUMBER of 'BBBBBBBBB" it took us to web page with a URI containing 'QnnyBZ4_ZB6qvm=qqqqqqqqq'. There was obviously some type of character substitution cipher being used on the URL. The maximum size for an account number was 9 characters. But with a few queries I could figure out the entire character set mapping. I searched for an ACCOUNT NUMBER of "ABCDEFGHI" and found a URI containing 'QnnyBZ4_ZB6qvm=QqnPvka03'. I searched for 'JKLMNOPQR' and found a URI containing 'QnnyBZ4_ZB6qvm=wMU6Zybjm'. I repeated this process for every upper, lower and numeric character and soon I had the following mapping of characters.

Normal Letters = ‘ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789’

Encrypted Letters = ‘QqnPvka03wMU6ZybjmK4BRSEWdVishgClpI1AouFNOJ9zrtL2Yef7Tc8GxDHX5’

Python makes translating between two sets of characters easy. Using Python 3 we can do the following:

 

blogimage1

 

This translated the word "HELLO" into "OvUUy" using the character mapping specified. The arguments for maketrans are the "FROM STRING" followed by the "TO STRING". Going in the other direction is simply a matter of reversing the parameters passed to str.maketrans() and passing (encrypted_letter, normal_letters)

 

blogimage2

 

In Python2 you have to import the string module because the maketrans function is stored there. But otherwise the syntax is the same.

blogimage3

 

Now I can decode the URLs! So we tried in on the URL we saw earlier.

blog1_NEW

 

Awesome. Now that is something I can understand. Now that we can freely encode and decode our attacks we had a bit more success with manual exploitation. But I’m lazy! I want SQLMAP to automate my attacks for me! If I don’t tell SQLMAP how to encode its injections it will not work against the website. SQLMAP tamper scripts are designed to do exactly that. SQLMAP is distributed with a set of "TAMPER" scripts to perform tasks like add a NULL byte to the end of injections or randomize the case of the letters in your query.  Creating a custom tamper script to do our character transposition is pretty simple. SQLMAP is using Python2 so we will have to import the string module.   But Looking at one of the other tamper scripts and using it as an example we quickly wrote the following:

 

tamper-script_new

We saved this new file a “custom_caesar.py” and placed it inside SQLMAP’s “tamper” directory. Then we pass the name of our script to the –tamper argument.

python sqlmap.py -u “https://www.example.tgt/webapp.php? QnnyBZ4_ZB6qvm=xxxTcTc&k3mK4_ZQ6v=6V9A&UQK4_ZQ6v=qVllgrr” –tamper=custom_caesar.py –dump

 

Then sit back and watch SQLMAP’s barrage of winning. A few lines of custom Python code took this vulnerability from “an unexploitable false positive” to a significant vulnerability that requires immediate attention. After using the tamper script we are able to access everything in the database with SQLMAP saving us hours of manual exploitation and encoding. And all it took was plugging 3 lines of custom Python code into an existing tamper script template.

Python is awesome and having the ability to use it and customize tools to meet your demands is incredibly powerful. Come check out SEC573: Automating Information Security with Python.

The SQLMAP Tamper Script

https://gist.github.com/MarkBaggett/49aca627205aebaa2be1811511dbc422#file-custom_caesar-py

  • Mark Baggett

 

Upcoming SANS Special Event – 2018 Holiday Hack Challenge

KringleCon

SANS Holiday Hack Challenge – KringleCon 2018

  • Free SANS Online Capture-the-Flag Challenge
  • Our annual gift to the entire Information Security Industry
  • Designed for novice to advanced InfoSec professionals
  • Fun for the whole family!!
  • Build and hone your skills in a fun and festive roleplaying like video game, by the makers of SANS NetWars
  • Learn more: www.kringlecon.com
  • Play previous versions from free 24/7/365: www.holidayhackchallenge.com

Player Feedback!

  • “On to level 4 of the #holidayhackchallenge. Thanks again @edskoudis / @SANSPenTest team.” – @mikehodges
  • “#SANSHolidayHack Confession – I have never used python or scapy before. I got started with both today because of this game! Yay!” – @tww2b
  • “Happiness is watching my 12 yo meet @edskoudis at the end of #SANSHolidayHack quest. Now the gnomes #ProudHackerPapa” – @dnlongen
kringle_02

How to Guide: Cracking into Piles of Files

by: Matt Edmondson

(Editor’s Note: this blog was originally submitted for posting on March 25th, but wasn’t posted until October 4th. We hope you enjoy this content and that Matt Edmondson continues to give freely of his knowledge to the blog.)

Password cracking is one of my favorite parts of information security. Many of us have built computer systems designed specifically for this task. We use tools like John the Ripper for quickie password cracking efforts and Hashcat for larger tasks but we usually sick these tools on cryptographic password hashes we obtain from hashdumps, database tables and tools such as Responder. One of the lessons we’ve learned from high profile breaches such as Sony is that it’s extremely common for users and organizations to also store passwords in document formats such as Microsoft Word and Excel.

In this post we’ll talk about why we use video cards instead of our CPUs to crack passwords and how we can use these same tools and techniques to help us get into password protected office documents and archive files that may contain a user’s complete list of passwords or other juicy information.

John the Ripper is often the first tool that information security practitioners play with. It’s free, easy to use and even tries to automatically detect what type of hashes you’re attempting to crack. John is available for Windows, Linux and OS X and is built into several Linux distributions like Kali. If you’ve never used John before and are using a Windows system, you can download it from http://www.openwall.com/john/ . I would recommend the “jumbo” windows binaries. The jumbo version of John supports more cryptographic hash formats than the regular version of John does as well as a few extra features. The windows binaries require zero setup outside of just extracting the files to a directory.

There is a blog post at https://www.tunnelsup.com/getting-started-cracking-password-hashes/ which gives a very quick overview of using John and has links to some password hashes as well as wordlists for use in password cracking.

After you start cracking passwords, you’ll probably get a strong, strong desire to start cracking them faster. The most common solution to this is to use a system with one or more graphics cards (GPUs) and use the GPUs to crack passwords instead of the computer’s CPU. While the jumbo version of John does support GPUs, the most popular tool for GPU password cracking is Haschat. It’s very well supported, constantly updated, free and can be downloaded from https://hashcat.net/hashcat/ . As long as your video card drivers are up to date, Hashcat should automatically detect your GPU and use it for cracking.

Underneath the surface, password cracking requires a lot of math. Graphics cards are able to perform math functions like these at extremely high rates of speed. Because of this, a tool which utilizes GPUs can crack passwords MUCH faster than a CPU can. How much faster? I made a password protected .RAR file for a quick comparison.

Utilizing John the ripper and my CPU to crack the password, I was making 174 guesses per second. This may sound like a lot but for password cracking, this is as slow as molasses.

pwcrackingtalk_01

The laptop I’m typing this post on has an integrated graphics card. These types of cards are usually the least powerful type of GPU for these purposes. Even so, Hashcat was able to use the card to make 1,432 guesses per second or over eight times faster than the CPU.

pwcrackingtalk_02

With a powerful video card, the speed increase is usually extremely large. A GTX 1080 Ti video card (currently over $500 on Amazon) is able to make over 50,000 guesses a second which is almost 300 times faster than CPU alone.

pwcrackingtalk_03

Getting started using GPUs to crack passwords is easy, but not necessarily cheap. If you have a system with a powerful graphics card, you already have everything you need. If you have a desktop computer sitting around you can probably buy a graphics card to put in it but you may have to upgrade the system’s power supply as well as high end cards tend to be power hungry.

If you’re on a budget you can buy a midrange video card like a GTX 1060 and still get a lot of bang for your buck. If money is no object, you can chain together multiple video cards and crack even faster. The Hashcat forums at https://hashcat.net/forum/ are a great place to check for performance benchmarks and ask advice.

These are all great options but we’re burying the lead here. If we point Hashcat at a RAR file to crack, it will look at us confused and politely let us know that it doesn’t see any hashes in the RAR file. There are isolated tools out there that will try to crack specific file types but we didn’t build a GPU password cracking rig to not use its power to crack so how do we get a hash from the RAR file? The answer is John the Ripper. Well, not John himself, but his friends.

In the latest version of John the Ripper there are approximately sixty add on applications and python scripts in the run directory designed for you to be able to point at a file for it to generate a hash.

pwcrackingtalk_04

I’ve tried a few different file formats over the years and have had relatively good luck. Here is file from winrar:

pwcrackingtalk_05

From excel 2013 (not that anyone stores their passwords in an office document):

pwcrackingtalk_06

And a PDF:

pwcrackingtalk_07

Once we have the hash we can then use Hashcat or any other tool we want to try to crack the password. If you’re curious what the hash for a specific type of file should look like, the Hashcat wiki has a great entry of example hashes at https://hashcat.net/wiki/doku.php?id=example_hashes.

pwcrackingtalk_08

So while a lot of us have gone away from using John the Ripper in an effort to take advantage of the power of GPU password cracking, we can still use him when we’re trying to get into password protected files that we commonly encounter on engagements.

There is a lot of overlap between the different realms of information security and one of the things I love most about teaching the SEC504: Hacker Tools, Techniques, Exploits, and Incident Handling for SANS is getting a chance to look at techniques from both an offensive point of view as well as a defensive perspective. Utilizing a GPU to access password protected documents can provide the keys to the kingdom on a penetration test, or a vital piece of information while performing digital forensics work.

Matt Edmondson
SANS Instructor
Twitter: https://twitter.com/matt0177

 

Upcoming SANS Special Event – 2018 Holiday Hack Challenge

KringleCon

SANS Holiday Hack Challenge – KringleCon 2018

  • Free SANS Online Capture-the-Flag Challenge
  • Our annual gift to the entire Information Security Industry
  • Designed for novice to advanced InfoSec professionals
  • Fun for the whole family!!
  • Build and hone your skills in a fun and festive roleplaying like video game, by the makers of SANS NetWars
  • Learn more: www.kringlecon.com
  • Play previous versions from free 24/7/365: www.holidayhackchallenge.com

Player Feedback!

  • “On to level 4 of the #holidayhackchallenge. Thanks again @edskoudis / @SANSPenTest team.” – @mikehodges
  • “#SANSHolidayHack Confession – I have never used python or scapy before. I got started with both today because of this game! Yay!” – @tww2b
  • “Happiness is watching my 12 yo meet @edskoudis at the end of #SANSHolidayHack quest. Now the gnomes #ProudHackerPapa” – @dnlongen
kringle_02