Saturday, March 16, 2019

I can Linux and So Can You (Bash commands) pt.2

So we can navigate and manage files from Bash. Now let's look at how to view, edit, and alter file permissions. So let's get started.

First, let's create a directory to work in. For that we can use the command mkdir. Let's also keep in the Documents folder.

$ cd Documents
$ mkdir testing
$ cd testing

So let's create a blank file. There are a few ways to do this. There is the touch command and simply writing nothing to a file name. The touch command is used for some extra stuff, so let's start with just putting nothing to a file.

$ : > test

If you ls to view the files, you should see one called test. The : is a minimal function that does nothing. As a result, you then send the output, or lack there of, to a file. Now making a blank file doesn't really do much, so let's write something to the file.

$ echo "This is a test" >> test

This appends the string "This is a test" without the quotes to the file test. The difference between > and >> is that > write to the file overwriting the original and >> will append to the file. To quickly view this file, we use the cat command.

$ cat test
This is a test

Okay, so if we want to overwrite the file and replace it with something, we use > and if we want to clear it all we use : > and rm if we want to just delete it. So how about we look at editing a file.

On Linux systems, and a handful of others, vi is usually included. In fewer cases, vim is included. If you have vim, I'd recommend that, but for now I'll just go over some basic vi commands. The vi editor is a visual editor that can be very powerful. The problem often is that it takes some getting used to as it's not like most programs people use today. Let's open a blank file and make a quick script.

$ vi hello.sh

It should dump you into the vi editor. To type into the editor, we need to enter insert mode. For this we press the letter i. Type in the following.

#!/bin/bash
echo "Hello, world!"

Now let's save and exit. First press esc so we can enter a command. After that we start the command with : and we want to save, which is w, and quit, which is q. So we can type :wq and press enter. Now we have a file we can actually run, almost. If you want to run it right now, you can type in this.

$ bash hello.sh
Hello, world!

Okay, that's great, but let's make it so we can execute the file directly. For that, we need to learn the chmod command, or change mode. When we talk about modes, we are refering to the permissions for read, write, and execute. In this case, we are interested in execute permissions. The permissions are also broken down into owner, group, and everyone. So let's look at how that works.

User permissions can be represented by either a letter, or a number 0-7. Here's a basic rundown.

Execute: x and 1
Write: w and 2
Read: r and 4

To apply multiple modes with numbers, we simply add what we want together, for example:

--- is 0
--x is 1
-w- is 2
-wx is 3
r-- is 4
r-x is 5
rw- is 6
rwx is 7

I often times find that when setting up new stuff I use numbers and when simply modifying a singular permission to do something, like execute, I use letters. Now when using numbers, we will often use three numbers for the owner (u), group (g), and everyone else's (o) permissions in that order. So say we want to open up a file so that anyone and everyone can see, run, and even edit the file, we do it like so.

$ chmod 777 hello.sh

I'd recommend against that because it's bad practice. If you need to get back to normal, chances are it was 644, so we just put in like so.

$ chmod 644 hello.sh

We want to add just execute permissions. To accomplish this we can figure out the number (755) or we can add the execute permissions. To add permissions with a letter we use the + and the letter. To remove a permission we use - and the letter. So let's make it executeable.

$ chmod +x hello.sh

Now we can run the file as need be. That is simply by running them like so.

$ ./hello.sh
Hello, world!

Notice the ./ in front, that indicates we are running the script in the folder we are already in. Now if we do an ls -l we get something that looks like this.

$ ls -l
total 8.0K
-rwxr-xr-x 1 root root 34 Mar 15 21:28 hello.sh
-rw-r--r-- 1 root root 15 Mar 15 20:52 test

Notice the -rwxr-xr-x, this shows us our permissions. Now hold up a moment, what if we don't want everyone and their dog being able to run the script? Okay, let's remove the executable permission for everyone. Everyone will be o. So we remove it like so.

$ chmod o-x hello.sh
$ ls -l
total 8.0K
-rwxr-xr-- 1 root root 34 Mar 15 21:28 hello.sh
-rw-r--r-- 1 root root 15 Mar 15 20:52 test

Okay, so now everyone can just simply read that file. We can do even more complicated stuff like applying multiple modes, removing multiple modes and doing so to multiple sets of permissions by separating with commas. At that point, I'd rather just do some simple addition and figure out the numbers. If you want information you can always enter in this command.

$ chmod --help

That will print out the help information. For now, we got the basics. So let's get back to file editing. We could open up a text editor, find it, delete, then replace it, or we could do it with a command. While when doing it with a command is a bit blind, it's easily scriptable. For this we use the sed command. As with most commands, you can get more information with the --help option. So let's look at checking this out.

$ sed --help
$ sed -i 's/world/universe/' hello.sh
$ cat hello.sh
#!/bin/bash
echo "Hello, universe!"

So, that was pretty nice, but how does it work? Well, the -i option is the in place flag. This lets it know we want to edit the file and not just print the output to standard output. The second part is what is called a Regular Expression, or regex for short. Regular expressions are an extremely complicated subject that has a lot of power to give at the cost of potentially massive headaches. But understanding the basics are a good thing to know. For now I will forego the description and stick to the basic examples. The real use here is allowing the creation of automated scripts to configure things for you as most configuration files are text based in the wonderful world of Linux.

So what can we do with sed? Using the regex, here's a few options.

s/this/that/ substitutes this with that.
s/this/that/g does the same thing, but for every instance of this.
/this/d deletes this.
s/this//g deletes all instances of this.
/something/s/this/that/ replaces this with that on line that matches something
y/abc/def/ transforms abc to def
/something/y/abc/def/ transforms abc to def on line that matches something
/this/p prints all the lines matching the pattern

With good regex experience, you can accurately automate editing of files how you need, of course with an understanding of possible expectations. With too much reliance, you could have a world of headaches. As a good practice, before editing a file, you may want to copy it to keep an old version just in case. In fact, it's something you'd do enough that it's actually built into the command! So let's look at that. Let's change universe back to world, but keep the other one as a copy with the suffix .old.

$ sed -i.old 's/universe/world/' hello.sh
$ ls
hello.sh  hello.sh.old  test

If you want, cat each file and see the difference for yourself. Or you could run each one. Either way, we got the basics down without getting into regex, so let's move on.

In more complicated situations, you will probably want to do this manually instead of spending hours on end creating a regex pattern to match something you can do yourself in a minute. Or perhaps we want to add more stuff to our script. Let's go back to the great vi editor. Let's open our hello.sh

$ vi hello.sh

So let's say we want to jump down to the very last line. For that, we type a capital G. The cursor should have jumped to the bottom of the screen. Now let's insert a line in here, so press i. Let's use some of the commands we've already done. Then add the following lines.

echo "This is a test" > testing
echo "This is appended to testing" >> testing
sed 's/test/something/' testing

After that, let's save and exit. Now let's run it.

$ ./hello.sh
Hello, world!
This is a something
This is appended to somethinging

Well, now isn't that weird? We don't want that somethinging. Okay, let's fix this, for that we need to make sure we only do this on the first line. So let's vi this file again.

$ vi hello.sh

If you've been following along, we need to edit line number 5. You can either arrow down to the line or enter the command :5 to jump down, remember to press enter. Now we can either arrow over to the spot we want to edit, on the ' or we can press w. So either arrow their, or type :5<enter>w. Now instead of pressing i, we press a. It works like this, i is to insert on the cursor, a is after, I is the beginning of the line and A is the end of the line. So press a, then type a 1 for line 1, then escape.

Okay, now let's say we want to save this as a different name, like newhello.sh. For the we type in :w newhello.sh and press enter. After that we can :q to get out of here and go run our new script. But wait, it says we can't! That's because although we saved the file as a different name, we still have the original open. After we alter a file, we need to explicitly save or ignore the changes to leave. Let's ignore it with the command :q! and press enter.

Okay, now we're out, and want to run our new file. If you try, you will probably get a permission denied error. That is because this new file did not inherit the permissions of the original. Let's copy those over, since we already know how to do it manually.

$ chmod --reference=hello.sh newhello.sh
$ ./newhello.sh
Hello, world!
This is a something
This is appended to testing

Woohoo! We're getting things done now. So now let's quickly run through some vi/vim basics that should be helpful in the future, or else we could be here a while if we keep stopping along the way.

$ vi newhello.sh

Okay, let's work on this file for good measure. As mentioned before, G jumps us down to the bottom, so do that.
Now let's jump to the top, we do that by pressing gg, yes that's twice tapping g.
To jump to the end of the line, we press $. To jump the the beginning of the line, we use 0. If there is whitespace at the beginning, we can get to the beginning of the line of text with ^.
Arrows can move the cursor around in the same directions, or you can do h for left, j for down, k for up, and l for right.
Let's jump to line 2 with :2<enter>.
Pressing w jumps you forward to the beginning of the next word or punctuation and a capital W jumps you based on whitespace. Likewise, b and B go backwards of w. The letter e jumps you to the end of a word and a capital E jumps you to the end before whitespace. So experiment with that on line 2.
Now, let us get back to the beginning of the line. Let's highlight the word test. To do that we want to jump to beginning four words down. So we type 4W<enter>. Now we press v to enter visual mode, then e to jump to the end of the word. From here we can do a lot of things, let's start with copying the word. Press y for yank.
Okay, it's copied, let's hit p to paste. Wait, what? ttestest? That's not right, is it? Well, p pastes right after cursor. Let's undo that with u. to redo, it's ctrl+r, not just r.
Let's try that again, press capital P. There we go, that pastes before the cursor. Now let's just delete this. So press these keys, bvex. The x key is the delete key that will delete directly under the cursor or whatever is highlighted in visual mode. So let's put test back, after all, we copied it. Press P. Hmm... that says testtest and not test. Here's the thing, unless we dump what we deleted into oblivion, it copies. If you don't want to copy it, before we press x, we do "_ to indicate a "black hole register." So instead it would have been bve"_x.
Okay, let's just replace it instead. So type in bvec, it will delete the word and dump us into insert mode, so now we can just type in test.
Press escape and now let's talk about what r does. If you type in r, it will replace the character directly under the cursor. Press r and then any character you want and you can see it replace. It's actually something I use quite a lot. Now if in the previous command we did r instead of c, and tried to type test, we would have gotten tttttttt as soon as we hit t. Might be useful when changing a hex code to black or white.
So let's say we want to copy the line. Tap y twice, yy. After that, regardless of where the cursor is, tap p or P. Now we have two copies of that line. To delete that line (cut the line). If you don't want to cut the line, you can do "_dd like with x.
On the topic of whole lines, there is a line visual mode. For that, it's a capital V instead of lower case v. Tap V to select all the lines you want, and like with insert, esc to exit if you need.
Finally, let's talk about copying to the clipboard. Sure, you can highlight the terminal and copy the normal way. The problem is, then you lose the powerful and quick navigation when in visual mode. We just need to copy to the correct register. Remember the _ black hole register? Well, we want the + register. So before using y, yy, d, dd, or x, type "+. Likewise we can do "+ before p or P to paste from the clipboard.

Okay, so quick rundown of registers, you can use a bunch of them. It's the character directly after the double quote ("). If you want to view registers or see where things are being copied to, run the command :reg and it will bring up a list of all the registers in use.

This is just a broad view of vi, but it will work for now. So we've edited files with sed and vi, we've altered permissions with chmod, we've sent output to files, so what else could we mess with? Access times for files are tracked. Often times we can use these with various ways to incrementally backup data. Because of this, there is a command to alter this. To do this, we touch the file. Let's start by looking at that wonderful help menu (:q! to leave vi, if you forgot already).

$ touch --help

So there are a few things I see touch frequently used for, the most common being making a blank file.

$ touch blankfile
$ : > blankfile

Both of these accomplish the same thing, just one take a little less effort. However, as mentioned, we can alter the access time. Here's the basics.

$ ls -l
total 20
-rwxr-xr-- 1 root root 144 Mar 16 13:06 hello.sh
-rwxr-xr-- 1 root root  37 Mar 16 12:30 hello.sh.old
-rwxr-xr-- 1 root root 145 Mar 16 13:21 newhello.sh
-rw-r--r-- 1 root root  15 Mar 15 20:52 test
-rw-r--r-- 1 root root  43 Mar 16 13:31 testing
$ touch test
$ ls -l
total 20
-rwxr-xr-- 1 root root 144 Mar 16 13:06 hello.sh
-rwxr-xr-- 1 root root  37 Mar 16 12:30 hello.sh.old
-rwxr-xr-- 1 root root 145 Mar 16 13:21 newhello.sh
-rw-r--r-- 1 root root  15 Mar 16 18:44 test
-rw-r--r-- 1 root root  43 Mar 16 13:31 testing

Look at that, we altered access time. So let's try something else.

$ touch -t 7001010000.00 test
$ ls -l
total 20K
-rwxr-xr-- 1 root root 144 Mar 16 13:06 hello.sh
-rwxr-xr-- 1 root root  37 Mar 16 12:30 hello.sh.old
-rwxr-xr-- 1 root root 145 Mar 16 13:21 newhello.sh
-rw-r--r-- 1 root root  15 Jan  1  1970 test
-rw-r--r-- 1 root root  43 Mar 16 13:31 testing

Look at that? Isn't it... epoch? Just wanted to make that joke. Okay, so you can do a handful of things with this. There's an -a option, which keeps it to altering just access time and not the modification time. With the -t we can do a 2 year or 4 year date. In the case of this example, we use a 2 year date for 1970, the 01 for month, 01 for the day, 00 for the hours, 00 for minutes and .00 for seconds. There is also the -d option that can take a date string, but the numbers are a little easier. The -m option alters just the modification time.

Another thing to mention is that the -c option to prevent creating a file if no file exists. If you may accidentally create an artifact because of a goof or something, might want to consider that as an option.

These are just some of the basics of viewing, editing, and altering files. There are plenty more, especially if you have selinux enabled. For now, this should get you going for a lot of Linux stuff. Exploring basics of regular expressions and some more uses of vi and vim will be a great asset for the future. There are certainly many other tools to learn, but now we can do many things necessary to actually get to using a Linux system.

No comments:

Post a Comment

Tag Cloud

.NET (2) A+ (5) ad ds (1) addon (4) Android (4) anonymous functions (1) application (9) arduino (1) artificial intelligence (1) backup (1) bash (6) camera (2) certifications (3) comptia (5) css (2) customize (11) encryption (3) error (13) exploit (5) ftp (1) funny (4) gadget (4) games (3) GUI (5) hardware (16) haskell (6) help (14) HTML (3) imaging (2) irc (1) it (1) java (2) javascript (13) jobs (1) Linux (19) lua (1) Mac (4) malware (1) math (6) msp (1) network (13) perl (2) php (3) plugin (2) powershell (8) privacy (2) programming (24) python (10) radio (2) regex (3) repair (2) security (16) sound (2) speakers (2) ssh (1) story (5) Techs from the Crypt (5) telnet (1) tools (13) troubleshooting (11) tutorial (9) Ubuntu (4) Unix (2) virtualization (2) web design (6) Windows (16) world of warcraft (1) wow (1) wx (1)