I Can Linux and So Can You (bash commands) pt.3

Previously we worked on viewing and editing files using cat, vi, and sed. Now let's talk about searching and reporting. While it would be simpler and easier to learn Python or Perl for these purposes, we are instead going to discuss grep, awk, and some basic regex. Despite the awkwardness of awk and the accessibility of regex for programming languages like Perl and Python, awk is still utilized by many. Other variations also exist such as gawk (gnu awk) and nawk (new awk), but awk works for both and generally is shipped with the box. So I will stick to that for now. First through, I think a discussion of regex is in order. So buckle up, because it's gonna get bumpy.

Sidebar, you can test these out with grep real quick. Simply do this:

$ echo 'The quick brown fox jumped over the lazy dog.' > test.txt

Then run the regex without the outside slashes like so:

$ grep --color -E 'regex here' test.txt
or
$ egrep --color 'regex here' test.txt

Regular expressions use lots of special syntax for searching and grouping complex text. Often times people will be familiar with some of it due to the commonality of certain methods. I'm sure most are familiar with "wildcards" (*). There are also some different types of regular expressions, but I will try to avoid specific things and keep this as generic as possible. So let's start with a sentence to search.

The quick brown fox jumped over the lazy dog.

Every letter in the alphabet there. Now let's assume we are trying to find this line. Let's work to match as much of this line as possible and cover as much as we can. To encapsulate all of it, I will keep it between two forward slashes because that's commonly how you will come across it. So let's talk about the start of the sentence. That can use a special character, in this case it's ^. There are many special characters ([\^$.|?*+(){}) which to match them litterall requires putting a backslash (\) in front of them. Next we want to match a capital letter. The thing we want is the nested character bracket ([) for this. This allows us to search a grouping of characters in a single position, for example vowels would be [aeiou], capitals would be [A-Z], numbers would be [0-9] or \d, lower case would be [a-z], alphanumeric would be [A-Za-z0-9] or \w. Okay, so let's make magic.

/^[A-Z]/

Okay, that will only match a string that begins with a capital letter. Now let's narrow it a little more. Next we want to match 2 non-space characters and a space. To match a non-space character we use \S and to match spaces (space and tab) is \s or a literal space is a space. Or we can do \w for a word character, or \l for a lower case. So all of these work.

/^[A-Z]\S\S\s/
/^[A-Z]\w\w /
Confused yet? Yeah, it's complicated with many ways to do the same thing, but they are slightly different. Okay, so now let's talk about the next word. It's five lowercase letters. We can do this pretty easy, there's a way to look for a pattern of n length or between n and m characters long. We do something like [a-z]{5} for five characters or something like say... [a-z]{2,5} would be between 2 and 5 characters long. So let's do the 5.

/^[A-Z]\S\w\s[a-z]{5} /

Okay, so now we have the word brown. Let's assume we don't know the length of the word we are matching, just that it's made of word characters. We know it's at least one character long. The plus (+) character comes into play here. A search plus a + will match 1 or more instances of it. So \w+ matches brown, \d+ matches 8675309 or 1.

/^[A-Z]\S\w\s[a-z]{5} \w+/

Can we get more complicated? Yes we can. We know the next word is fox and a space. Let's match literally anything after this of 0 or more characters. For that we use the period (.) which matches one of anything and asterisk (*) which matches 0 or more of the pattern. If you wanted to do one or more, you would do .+ to achieve that. We'll add a space after, so it would either match two spaces or a word followed by a space. The catch is that it will match from the end of brown to the last instance of a space.

/^[A-Z]\S\w\s[a-z]{5} \w+ .* /

Okay, so two things left, the word dog and a period. Let's say we want to match either dog, or cat or neither? Well, we can do one or both. To do or we use the pipe character (|), and to keep it clean we will use a group, which goes in parentheses (()). To match 1 or 0 instances of something, the pattern gets followed by a question mark (?).

/^[A-Z]\S\w\s[a-z]{5} \w+ .* (dog|cat)?/

Okay, the light at the end of this tunnel is near. Or is that a train? No matter, into the breach! The sentence ends in a period. We need to match a period. Two things. First, since the period is a special character we need to escape it with a backslash (\). Second, the end is marked with a dollar sign ($), like in vi and vim.

/^[A-Z]\S\w\s[a-z]{5} \w+ .* (dog|cat)?\.$/

Now look at that. What a mess that is. Obviously we won't often need super complicated stuff like that. There's still a lot more, but this should help get you started. For a better reference you can check out this regex quick reference, and it also goes over the difference in types of regex. If you want a good amount of flexibility, Perl compatible regex is usually the way to go.

So now let's talk commands. Searching and reporting are two things computers should excel at. Most Linux distributions today come with egrep and gawk and there are also counterparts like grep and awk, which they are based on. You can use the -E option with grep or egrep. I suggest using the color option if it's not already aliased on your distribution so you can see what part matches. I'll cover aliases in another portion. The awk and gawk commands are part of a type of programming language made for searching and reporting. Usually you can turn to Perl or another programming language, but awk works for a quick and dirty one-liner.

So let's start with searching. The grep and egrep commands can be used to search files or output of other commands. When you need to narrow your output to a readable level, this will be the go-to. I often times just use grep because I'm just searching for a word and not some expressions, but we'll check out both. I think a good populated folder to use for demonstration will be dev, there are consistencies in there. So let's take a look.

To get an idea of how many files are in dev, take a moment to just look.

$ ls /dev

I'm not going to post mine because that's a long list. Now let's say we are trying to find if a partition exists on our hard drive. Well, it's listed in here. We know our drive is sda, so how do we list all the sda drives? Well, we pipe the output from ls through grep to do a search.

$ ls /dev | grep 'sda'
sda
sda1
sda2
sda3
sda4
sda5
sda6

Cool, so here we see I have 6 partitions (The first three are Windows related, 4 is extended, 5 is kali, and 6 is swap). So let's assume we want to just make a cut-and-paste command to search for hard drives in other older systems. Some use hda instead of sda. Let's also assume we want to check for multiple hard drives, so there could be an sdb or an hdb. Easy.

$ ls /dev | grep -E '^(h|s)d[a-z]'

If we don't include the ^ at the beginning I get watchdog in the result for the hdo part. That's really all there is to it for simple searches of output. We can also search through the files of entire directories with the -r option.

Beyond search filters, it's often necessary to report the findings. Often times that reporting or searching will be useful to dump into an actual readable format. For now, let's use a custom file we will call searchtest.txt, here's what I put in it.

id:title:author
1:The Origins of Modern Science:Herbert Butterfield
2:Catch 22:Joseph Heller
3:1984:George Orwell
4:Animal Farm:George Orwell

I made a column title as well. So let's run a quick search.

$ grep -i 'george' searchtest.txt
3:1984:George Orwell
4:Animal Farm:George Orwell

The -i option tells it to ignore the case. Now this is all fine and dandy, but still a bit difficult to read. So let's clean up the reporting phase using awk. We'll discard the id and just print the title and author.

$ awk -v FS=: '{print $2,$3}' searchtest.txt
title author
The Origins of Modern Science Herbert Butterfield
Catch 22 Joseph Heller
1984 George Orwell
Animal Farm George Orwell

Okay, so we have a lot going on here. We are using -v to set some variables, in this case the Field Separator (FS) from the default of whitespace to colon (:). This allows us to access fields by a dollar sign ($) followed by the place number. In the case of this file, we have $1, $2, and $3. We then have an actual code block where we print fields $2 and $3 separated by the Output Field separator (OFS), which defaults to a space. Okay, this is still messy. We could alter the OFS to make it clearer. There is another method to format your print statements a little better, printf.

$ awk -v FS=: '{printf "%-29s|%s\n", $2, $3}' searchtest.txt
title                        |author
The Origins of Modern Science|Herbert Butterfield
Catch 22                     |Joseph Heller
1984                         |George Orwell
Animal Farm                  |George Orwell

Okay, so now we can actually read it. Here's how the printf is working. The %s is a string we're going to substitute in, in order of the arguments passed to printf. The %-29s is to make sure the string is padded to a length of 29 characters long and the - makes it align to the left, default without the - is to the right. Now let's say we want to not include that first line. We can add a filter for that.

$ awk -v FS=: '/^[0-9]/ {printf "%-29s|%s\n", $2, $3}' searchtest.txt
The Origins of Modern Science|Herbert Butterfield
Catch 22                     |Joseph Heller
1984                         |George Orwell
Animal Farm                  |George Orwell

So there we can see that it accepts a regex filter. You can use this to filter through, select, and print out reports of any text files you have. Learning more about printf will also all you to do a lot of formatting on the reporting as well. The awk and gawk commands happen to be programming languages in themselves as well, but going into all of that detail right now would be lengthy. So for now, let's discuss a bit more for the ins and outs of printf.

The printf function exists in many programming languages and even as a Bash command. You may be wondering what %s means. Well, the % indicates a control and the s indicates a string. In this case, it's printing a string. If you wanted to simply print a percent, you'd have to type %%. You also have %c, which prints a single character ascii decimal value, %d and %i which print numbers, %e and %E print a number in scientific notation, %f and %F for floating point numbers, %g and %G which print in either scientific notation or floating point (whichever takes fewer characters), %o prints numbers in octal, %u prints unsigned integers, and %x and %X print in hexadecimal where %X prints in uppercase and %x uses lower case.

Formatting modifiers can be added to it as well, like the - justifies to the left, default is to the right. A + tells it to print positive or negative signs. A number indicates the space it should use at minimum and a decimal, like 5.2 would indicate a 5 character width with a floating point precision of 2. A leading zero will pad a number with zeros instead of spaces. A # tells it to use an alternate form for certain numbers, like hexadecimal 0x prefix. A ' will print numbers in the thousands to use a comma separator. As you can see, most of the formatting revolves around numbers, but the main thing is setting a width lets you put your output in columns, which can make it easier to read.

So for example, if you did a printf with %07.2 to 12.3, you'd get

0012.30

It's seven characters wide, padded at the beginning with zeros and held to a precision of two.

For now that's enough, next I will cover applying the regex to searching and substituting with vi and sed.

Job Hunting: Finding a Tech Job, Resume, Cover Letter, and Interviewing

Over time of applying to various jobs, I've slowly refined my Resume, Cover Letter, and Interviewing techniques. I have searched online for how to write and organize everything to do as well as I possibly can, but the problem is that most things I find online are fairly general purpose. Each industry requires a little bit of a different technique, but I also find that depending on your experience and education it can also influence the way you want to write it and present yourself.

So let's start with the resume portion first. A tech resume can be generalized or specialized, but I'd recommend keeping a few copies for specialized cases. For example, if one job puts more emphasis on networking, that should be shown first, whereas another job may want more emphasis on server administration. So let's start with what sections the resume should have.

There should be a technical skills, education, professional experience, references, and a short description of yourself and abilities. Each of these may have different amounts of information and that can influence the order you put everything in. Also, you should try to order information by whats more relevant for the job and leave out any irrelevant fluff. The exception on removing irrelevant information is with previous job experience, feel free to add any jobs you have done to keep at least three. If you don't have any professional experience, there are other things we can do, but let's get to that later.

So at the beginning of your resume, you want to put your name, phone number, email, and address. Following that, you want a little blurb. For this, you want to basically write an advertisement for yourself. I always had difficulty with this because I can be a very blunt person and even more to the point when discussing myself.

Here's the one I use in my current resume for IT Specialist positions:

IT specialist with skills for repair, networking, and administration. Experienced with working across multiple locations and with many people. Great time management and people skills from working jobs that require traveling across multiple locations.

One thing I avoid is self reference. It's implied, so I just simply avoid it. Remember that people reading these can be going pretty fast, so I just simply avoid it. Not to mention it's redundant. It might not be best practice, but it works. The first sentence is stating basically me with everything the job wants. After that, I attempt to sell myself as being great at a job.

The next thing we need to look at is what order to put the sections in. One of the things that will drive this is how well the information in these topics will present you. For example, if you have little to no education, technical skills would be a better thing to present first. Job experience may be what you want to present first if you have a rich job history. I, personally, only have an AAS in CIS and some spotty job experience because I'm a free spirit! Okay, I just have a lot of issues finding a job due to life and bad choices. However, I do have some useful technical skills. So for me, I do technical skills, then education because I don't want that to be forgotten about, and then job experience.

So I'm going to follow mine in order for now. Technical skills. Most resumes say skills. We put technical skills because technology. The idea here is we put down job related stuff we can do. It will often echo the requirements and tasks listed in the job description. You should keep this limited in scope, 4 to 5 points is the better range to keep it in.

Another thing I put in mine as the first bullet point are my certifications. Certifications are a magical double that count as a demonstration as both skills and education. If you put education first, you can look at putting certifications in there as they are also formal training. I also make a point to write it with the company related name each time.

Another important point when writing these is to use keywords that demonstrate points of the job you would be performing. One of the big players is troubleshooting. Here's what mine look like at the moment.

• Certified with Comptia CySA+, Comptia Security+, Comptia Network+, and Comptia A+
• Troubleshooting experience for hardware and software problems as well as server and firewall configurations
• Networking configuration and cabling on both small scale single building and large scale multiple building setups
• Hardware repairs such as screens and hardware installations such as access points and projectors
• Experience with many operating systems including Linux, Windows Servers, Windows 10, and Mac OSX including for enterprise systems

So that's my short list of not-so impressive skills. Note that for the certifications, I put them in a reverse order of skill, the higher ones first. Everything should be relatively short and to the point. You can also put in things that are things you do outside of a job but that is still relevant.

Now let's talk the education section. Education is simply showing what education you have. It should include the institute, location, year of completion, and the degree or certificate achieved. Some pieces of information can be left out if necessary. For me, there is just one entry. It says my college, the city and state it's in, my degree and the year I graduated.

If you are still in college, you will want to include some of your courses that are relevant and current GPA. This can supplement some of your professional experience if you have a lack of relevant information to put in it. It's a pretty simple and straight-forward thing to write. You can also use the course description to write the description or directly copy and paste it. That's what I did for my resume when applying for an internship.

Next we talk about Professional Experience. This is one of those sections that has a good chance of having information that's not relevant. The reason is you want at least 3 but no more than five previous jobs listed. Even if a job is not relevant, spin whatever you can to something vaguely what they might be looking for. You highlight job duties that best represent a skill or task that the job requires. Here's an example of some of my job duties.

• Desktop support for hardware, software, and peripherals
• Network management including cabling and device configurations
• Creating and maintaining images for large-scale deployments for entire labs and even buildings
• Server configuration and management for Windows 2012 and CentOS used for computer imaging services

I tried my best to put in words that I find in job descriptions I apply for and other pieces of information that someone scanning the resume might be looking for.

At the very end goes your references. You should have three to five. Three to five tends to be the magic number.

Okay, so you have your resume, how long is it? One page is usually good, hopefully it's one full page. You can go onto a second page, but do not fill the page. Mine is two pages, but the second page is just references.

Now something that is super important and often overlooked, the cover letter. I often forget or am too lazy to write this on most applications. The funny thing is, all my calls I get are ones I specifically wrote a cover letter. I have come across plenty of cover letter cookie cutters. Do NOT use these. I made that mistake before and often realized after sending one that I forgot to change things here and there or that with some job titles it just read really strange. Not to mention it was rather robotic and just not something someone reading would consider.

Let's talk about writing one from start to finish. It may be a lot of work, but the payoff is worth it.

First we start with the obligatory addressing of "To whom it may concern." For the body you start with a simple introduction. That will consist of who you are and why you are looking at the job. If you already have a job, it may be worth explaining why you are leaving. For example, if you are looking for a job that's closer to home, that's fairly reasonable to why people may want to look for another job. If you want to avoid talking about your current job, then you can talk with more focus on the job you are applying for. You also want to in a sense echo the blurb you wrote, but in a more formal letter style. Make sure to highlight any key strength that they would want. Then to finish it off strong, tell them about how much you want to benefit them.
That is pretty generalized. Let me show you my most recent with some information censored out.
My Name is Matthew DeSantis. I am currently employed by ********** and trying to find a good job close to home. I have enjoyed working for the school systems and would like to continue to do so. I am familiar with the settings, requirements, and technologies that schools use. Currently I maintain three different schools (**********, **********, and **********) and have become accustomed to balancing different schedules of multiple places and assigning priorities to keep things as efficient as possible. As much as I love my current job, working for a different county comes with its own challenges and I would like to at the very least contribute to something closer. I would love to have the chance to contribute more to ********** and demonstrate what I can do and how I can be an asset.

Okay, so it's rather poorly written, but it was the best I could do at the time. I hope that helps demonstrate what a cover letter that isn't a cookie cutter might be like.

 Moving forward. You got a call back, now you have an interview. A lot of places now start with phone interviews to filter people out. Phone interviews are easy. It's mostly just questions and answers. I did one recently, and it was probably the easiest thing I've ever done. It started out with being asked for some basic information about myself and since I already worked the same job but for a different area, I brought that up to build a connection. You want to build a connection with the person quickly so they remember what you say. Try talking about a similar experience someone on your job might have to make this "one of us" sentiment. In the questions, be confident in your answers, but if you don't know do not waste the time of you or the interviewer. Be honest that you don't know and if you have a guess, let them know what you think the answer is. I had all the answers, but my sarcastic nature lead me to make a few jokes before actually answering. If you're like me, just remember to keep the jokes on the safe side.

Let's talk in-person interviews. In person interviews are always awkward for me for a few reasons that I'm sure others can associate with. I have some social anxiety, I'm a naturally sarcastic person, I can be pessimistic and opinionated, and I talk with my hands. In person, you should be aware of what you're doing and I sometimes lack that awareness.

So now what do you do for the interview? Let's talk about one I was in recently. I sat down with multiple people, as seems to be pretty common now for some reason. It started with a simple written test for technical skills. These consisted of some IP and subnet knowledge and some simple hardware knowledge.

After that we went on to questions. One of the questions I got asked was about what I was like and my job. This is one of the questions I often struggle with because it's so broad and open-ended. I tend to just describe my work history and work load. If you don't have much, try to think of something that might be relatable. After a handful of generic questions like what are your strong points and weak points, it gets to technical questions. These ones are easy for me.

I was asked about experience with Active Directory, which most places just expect a simple "yes, I can add, delete, and move around" stuff. Even if you've never done it before, it's nothing a quick Google search can't prepare you for. I was asked about my experience with different types of software and it's easy enough to rattle off everything I use at work or in my free time. I was asked about hardware repair as well. Most will ask about basic networking as far as do you know what the types of equipment are, how to plug them up, basic setup, really anything someone who is technical minded should be able to at least figure out with access to Google. I was also asked about my favorite and least favorite technologies. I'm not really one for a favorite, but least favorite is always printers, and with good reason. Printers use a combination of mechanical and computerized parts opening them up for many points of failure. Only other question I remember is about what to do if a computer is dropped from Active Directory. My solution is usually to boot it up in Clonezilla, unlock the Administrator account, then just rejoin it like normal.

After the line of questioning, I was given a couple of hands on tasks. The first was to find the mac address for a computer logged into windows. I was going to look through the network and sharing center, but decided ipconfig /all on the command line would be quicker and just did that. After that, they had a computer setup as broken to be fixed and try to get it working again. I had a hunch exactly what was wrong, but for due diligence, I decided to go through the diagnostic steps as well. It was simply just stuff being unhooked internally like the RAM and HDD, it wanted the video card used instead of the onboard. Nothing a simple step by step process couldn't solve in a few minutes.

All in all, if you've done this stuff before, it should be easy. The problem is when you've never done it before. My first time interviewing for the job I had now was a disaster. I made it all the way up to the in-person interview and it felt like it was all falling apart. After all the questions, they wanted a practical demonstration. So they handed me a list of tasks, set me at a laptop, and said have at it. I was allowed to Google and everything. Honestly, I should have been more apt to look up what I didn't know, and it was most of it. I actually ran the clock on the interview up to the last second to finish it. Here's the reason why. I had never done it before and I didn't do any research before going their on what I needed to know. When you apply to a job, you are given a list of job duties and responsibilities. Use that to your advantage and fill in any gaps before you get there.

All that being said, I hope this helps someone in their job hunt. Don't be discouraged if you have yet to find one, the job market for IT is very saturated at the time of writing this. It seems like everyone has at least some IT experience. If you want good areas to look at that have a good chance for getting your first job, while not exactly paying the best but still good enough, consider looking at schools and hospitals. Virtually every place has some IT staff, so make sure to look at other places than strictly computer places. I have also found a good handful of car dealerships desperate for IT staff.