The 90 Day Cyber Boot Camp

(The Agony of Hiring Cyber Talent — PART 3)

Parts 1 and 2 of this series talked about the difficulties in hiring cyber talent and the validity of the certifications that often accompany the potential hires. In this final part of the series, I want to delve into how we approach training our new employees, from ground zero up to those with strong experience. This is our own hybrid boot camp, and what started out as a necessary experiment is quickly becoming a cornerstone of our business practice.

To begin with, there are some basic vetting questions that are prerequisite before we hire. For example, an opening question to test the candidate’s ability to think on the spot might be, “How do you build a botnet?” This gets them to work out how they would infect, control, and coordinate a botnet from scratch; putting them in the mindset of the attacker. Then they may be asked, “How would you defend against your botnet?” to garner the blue team perspective.

Other basic questions a candidate can expect are:

• What are some ways malware can evade detection by antivirus products?
• What is a cross-site scripting (XSS) attack, and how does it work?
• Outside of XSS, what are a few other examples of web application attacks?
• What is a man-in-the-middle attack, and how can it be prevented?
• What is the difference between TCP and UDP? What kind of use cases are better for UDP? (fun fact — this very question helped us expose some areas that weren’t as strong as they were first presented to us.)

But again, these are just basic questions that anyone hiring a cybersecurity person should be asking to flesh out general knowledge and to weigh the creativity and expediency in the answers. They are more questions designed to fill seats, not roles.

However, every company has specific needs and proprietary systems, and book-taught cut n’ paste answers are often not sufficient. Our approach, our “boot camp” if you will, delves into specific skills around operating systems, common network protocols, internet infrastructures (i.e. DNS, PKI), basic scripting in bash/python/etc. technical writing, operations and client-facing communication relevant to our line of work.

Our 90-Day Boot Camp

We recently onboarded a security analyst who has some certified training, but most importantly, has a strong passion for all (well, most — not too keen on aligning div blocks in Internet Explorer 7) things cybersecurity and a general desire to learn.

We allotted 90 days to complete the boot camp and gave the freedom of self-pacing whilst providing support and guidance every step of the way. The bulk of the “course” was completed well under the timeline.

Now before we put the new hire out to sea to sink or swim, we did throw out a life preserver vest by providing direction to relevant online learning courses. Some of these included:

• Codecademy — Python
• Udemy — Blue Team Boot Camp: Defending Against Hackers
• ProfessorMesser.com — CompTIA — Network+
• Mitre ATT&CK

The following is a high-level overview of a sampling of our “boot camp” specifically catered to this particular new hire, including assignments, some of the processes for completing the tasks, and some of the feedback received from all levels of our organization during the training.

Training Tasks

1. Installing and setting up a virtual machine
2. Deploying and configuring a web/mail server
3. Creating a bash/python script for ease of automation
4. Making a self-hosting DNS
5. Setting up firewalls (i.e. iptables, shorewall),and then using nmap to test the firewall settings
6. Testing uplink performance (with various benchmarking tools)

Web/Mail Server -

1. Purchase a brand new domain name
2. Set up a new Digital Ocean box (including SSH hardening to only allow access with a public/private key), point the domain name to the server and create a simple webpage
3. Install nginx — configure the server to use the purchased domain name, set up an SSL certificate
4. Add an image/text onto the webpage
5. Add A/MX/PTR records to domain name
6. Install latest version of iRedMail.tar.gz, then configure iRedMail
7. Set up DKIM, SPF and DMARC records to secure and properly authenticate outgoing emails

Bash/Python Script

Create script using Bash and Python to automate running a load of tests to analyse internet performance and return those results inside a CSV file.

Self-hosted DNS

https://www.linuxbabe.com/ubuntu/set-up-authoritative-dns-server-ubuntu-18-04-bind9 — (Ensure 2 boxes are created for each DNS server — master & back-up)

1. Install all the necessary dependencies
2. Setup the OS using similar hardening steps to the above
3. Add a zone for a master DNS server, allow transfer to the IP address of the back-up DNS server
4. Update firewall settings
5. Add a zone for the back-up DNS server, allow transfer to the IP address of the master
6. Re-update firewall settings
7. Change glue records on domain registrar’s website (change NS records)

Firewalls (iptables, Shorewall)

iptables: https://upcloud.com/community/tutorials/configure-iptables-debian — iptables are a great way to practise with firewalls, but be prepared to break a load of boxes accidentally blocking ssh connection to the server. Edit these in a way that still allows access to ssh, http(s) but only through your own IP address and everyone else is blocked.

Replicate the iptables configuration but this time using shorewall. Again, edit in a way specific for your needs, but ensure ssh, http(s) connections and drop everything else.

Testing Internet Performance (benchmarking)

There are many tools for this, one being ab (apache bench).

1. Create a web server to test bandwidth.
2. Test to see how many requests are successful compared to how many failed. (Other benchmarks may vary on how results are shown).

Processes and Feedback

From our CTO:

This is a MITRE ATT&CK matrix. Use the common tactics adversaries will use to breach/attack, with the techniques that have been known to have been used. Focus on the highlighted ones as they are most relevant to us. NOTE: Monitoring these in real time is important because just in case they show up again, the good guys will have better knowledge of what the adversaries may do, that way they have better preparation to combat them (or to try and prevent them from entering an environment).

Or, this could just be used as a general guidance on what tactics/techniques may be most important when beginning to build up on security for any starting company, with the ability to gradually expand the protection as and when needed.

You can use navigation if 2 or more ads are popping up, you can see what hackers’ common approaches are, and if they use the same ones. If they do that, you can narrow down the list of priority for a faster, more stable security defence. Also use the detection/mitigation guidelines we’ve supplied to ascertain what the best approach(es) may be.

Training Submission Ex.1:

Trainee

In this screenshot, I’m trying to use df[‘blah’].astype(int) to get rid of the decimals, so I’m still looking into that…

Our CEO

Yeah, you don’t need the decimals. Also, it needs to be all in one file, not separate ones. For example, add a new column that says “document_size” or something like that.

Also, we need “bytes per second”. If the tool doesn’t give one automatically you’ll need to calculate it yourself.

Training Submission Ex.2

Trainee

Anything else that needs doing on this?

Our CEO:

It’s the number of threads that you want to be changing: 10, 25, 50, 75, 100, 150, 200, 250, 300, 350, 400, 450, 500 threads.

Training Submission Ex.3

Trainee

I tried pathlib, variable_name(open(path), pandas pd.read_csv.

Engineer 1

iirc, open()can only open a single path and you can’t use globs. The issue is that you are opening a file that doesn’t exist. When you are passing an argument to the with open(), you need to specify the full path to the file.

If you want globbing support then use from glob import glob and then glob(“/path/to/*.csv)and it will give you a list of files. Then you need to open(“path/to/file”)individually.

Trainee

So I need to list out each file individually in order to append it?

Engineer 2

Before opening the file, you will need to list all files inside your data/ directory. Save them to a list, write a “for” loop which will iterate across all items in this list, then open the files one-by-one. In order to get the list of files, you can use the function we typed above.

You can also use os.walkdir but it’s more complicated. In that case, I’d suggest using glob.glob.

os.walkdir is a nice route for traversing file system trees, like when you need to descend into multiple nested directories to do something with the files in there. Iirc, there is also os.list(). Also note that your need to read the data first, save it to a list (in Python) and then open a new file (eg. results.csv) in “write mode”. Write each line from this list to this file.

Trainee

I read that ‘r+’ can read and write at the same time, so I’m not sure if that’s what is making it append in the same file.

CTO

That’s true, but I doubt it’s what you actually want, with “iterator invalidation” being a thing.

Trainee

Maybe changing it to ‘w’ could overwrite the whole list and have only the criticals showing?

CTO

You’d be reading from, and writing to, the same file at the same time. I’d bet decent money that you’d have a sad…

What you probably want to do is move “writer” out of the loop and give it a specific path. That way, you’d be collating the results into a single output file, which is perfect for review!

The Bottom Line

I know, I know, the above script wouldn’t exactly get Hollywood into a bidding war, but hopefully you get the gist of our process: open communication and engagement from everyone to make the training as smooth and as relevant as possible. Tailored training will vastly help ensure productivity and the understanding of technical security needs for your company/organization.

As a startup it’s often necessary to cut some corners on the expenditures. It’s a necessary evil; an early survival tactic that keeps a company afloat through the turbulent waters of competitive marketplace positioning. But the one area a cybersecurity company, or any company with the need for serious IT (which, in my opinion, is every company that handles sensitive data in any capacity) can’t skimp on is proper security training. The investment it takes will almost certainly save money, reputation and your data down the line.

For more details/clarifications on our boot camp or anything else related to our company (or just drop a line to say “hi”), hit me up in the comments.